Jérémie Sublime

My research main research area is Machine Learning in weakly supervised and unsupervised settings. In other word, I specialized in designing Machine Learnign and Deep Learning methods that can process data with little to no prior knowledge with the goal of finding interesting structures and patterns that are relevant to real life applications. Examples of such applications include the fields of Remote Sensing and Medicine, where huge amounts of data of all kinds are available and waiting to be process, but only a very small amount of them is reliably labeled and can be processed by mainstream Deep Learning methods that are trendy nowadays, but requires huge amounts of labeled data to work properly.

Within this context, people come to me with all sorts of applications when regular off the shelf methods have failed. My job is then to use all the known tricks and new ones from the unsupervised learning book (clustering, change detection, autoencoders, data augmentation, transfer learning, data preprocessing, etc.) in order to provide them with an artificial intelligence solution that answers their needs with the utmost level of accuracy.

In short, most of my research fall into either of the following 2 categories :

Theory of unsupervised and weakly supervised learning
Applications of unsupervised and weakly supervised learning to medical and remote sensing data

Weakly supervised segmentation of the Brain stem [2023-2026]

Context: Seoyoung Oh PhD Thesis and internship, Sarah Heurtevent internship

Partner institutions: ISEP, INSERM, IHU de la Pitié Salpêtrière

The brain stem is a key structure that directly connects the brain and the spinal cord. As such, not only is it crossed by nerves going everywhere in the body, it is also the seat of key functions such as autonomous breathing and heartbeat. It is therefore a structure of great interest especially in the study of diseases such as Amyotrophic Lateral Sclerosis (ALS) or Parkinson's disease, and which is yet poorly known.

Within this context, our goal was to leverage various Deep Learning methods in order to provide a detailed segmentation of the brain stem and its structures. the challenges for this task are the following:

The brain stem is rarely the main focus of brain exams, and therefore available T1, T2 MRI and diffusion data are often of poor quality (contrast and distortion issues) when it comes to the area of the brain stem.
The brain stem is a somewhat small structure, and the structures to identify are for many of them only 1 to 5 pixels wide.
There are no annotated data to train a supervised neural network. The only known structures can be found medical atlases. This leaves us the choice between going fully unsupervised, or to give a shot at mapping the atlas data into the MRI space and hope that all patients have similar enough anatomy.
Our MRI data come from SLA patients which have a very short life expectancy after diagnosis. As such, most patients only have one and very rarely two scans available, thus making it difficult to use change detection to somewhat narrow down the potential areas of interest.

This project still is in its early stages, but we have already successfully proposed a deep learning method relying on mapping the atlases into the MRI space. This method relying on our proposed U-DecoNet and N-Deconet has proven to be good enough to map the main areas of the brain stem and was even able to detect some of the nuclei. Our current leads include the addition of salience map from patient-control classification using CAM, with the goal of highlighting areas of interest to segment with salience maps.

Oil and Gas Infrastructure identification from Methane detection [2021-2024]

Context: Jade Guisiano PhD Thesis

Partner institutions: ISEP, UNEP, IMEO, Université de Reims, Institut Polytechnique de Paris

With methane being one of the main source of green house gas emission, this project aims at providing an end-to-end Framework relying on satellite programs as well as Oil&Gas companies official data, with the goal of being able to link each detected methane cloud to its potential source.

This idea is displayed in the Figure below: It all starts with a geolocalized methane detection provided by one of the methane detection satellite program or company (Kayrros, GHGSat, Tropomi, etc.). The source must then be linked to a structure using regular imaging or radar data from any regular satellite program (Sentinel or ASTER for instance). Once the site has been identified, then next step is often to identify the faulty infrastructure among tanks, compressors and wells.

While this Framework seems quite straightforward, there are numerous issues to achieve this idea from end-to-end, both technical and political:

Methane detections programs are not live: It means that any detected cloud of methane might have been emitted 12 to 48h prior to its detection. It is therefore likely not above its original structure of emission anymore and might have drifted and diluted depending on weather conditions. As such linking a plume of methane to an infrastructure is very challenging.
Most of the Oil&Gas infrastructures are not geolocalized, and they tend to change often: In fact, mostly O&G infrastructures from the Permian Bassin in the USA are well geolocalized. It means that the task of linking a methane plume to an infrastructure first requires to detect the potential guilty infrastructures on a large number of images around the area where the plume was detected. These images can have varying resolutions, and infrastructures may vary slightly from an area to another, thus making it a very complex object detection problem with little to no labeled data (outside of the Permian bassin).
Most companies providing methane data are private, some of them funded by O&G companies. As such, data are often partial, potentially biased, and various NDA may prevent cross-validation with other programs in any of the steps of the proposed method.
All the data used for this problem are described in different referential, which makes it very challenging to link them to the same geolocalized source. Furthermore, they are of very different nature: A methane detection is just a set of coordinate with a volume and a reliability, O&G official data are text data, so are weather data, and the program used to identify the infrastructures provide either radar or image data.

Nevertheless, within this context my team was able to provide part of the first block which to any detected plum of methane can find the potentially faulty infrastructure based on optic images close to the likely source. We did so in a fully unsupervised way using various clustering algorithms. And we have also make great progresses with the second step of telling wells, from tanks and compressors in images of various resolutions and quality in order to further identify faulty structures.

Mapping of the Lamina Cribosa [2020-2024]

Context: Nan Ding PhD Thesis

Partner institutions: ISEP, IHU des 15-20

The Lamina Cribosa is a 3D mesh-like structure consisting of pores that allow the axons passing through from the retina to the brain. It has been identified as one of the primary site of damage in glaucoma the second leading cause of blindness in the world.

Within this context, our goal was to provide Deep Learning methods able to map the lamina cribosa from images extracted from 3D OCT volumes. Our goal was to detect the pores, in order to provide a precise map of the axons as shown on the Figure on the left.

The mains issues for this tasks are the following:

The lamina cribosa is a very small structure and the images we were provided with have lots of contrast and lighting issues (the example from the figure is one of our cleanest).
We have very few images to train an algorithm from
No annotations exist and even the best ophthalmologists often disagree on what is a pore with an axon inside of it and what is not.

To answer this problem, we proposed a complex framework that relied on several steps: First we used morphological and image processing methods (proposed by my colleague Pr. Florence Rossant) to improve the images and among other things increase the pore contrast. Then, working alongside practicians, we manually annotated a few of the pores everyone agreed on on some of the images. In a third step, we used data augmentation to increase both our image and annotation volume. And finally, we proposed a modified U-Net with attention mechanisms to focus on area of interest. Not only our proposed method was able to learn with very few annotated data, but it also proved robust to false negative annotations (the pores we missed). The results were checked by medical doctors who were satisfied with our proposal. In an ulterior proposition, we used pore continuity in the volume to further improve our results and to create maps such as the one you can see in this section with annotated pores and axons.

Unsupervised change detection and clustering in satellite image times series [2017-2020]

Context: Ekaterina Kalinicheva PhD Thesis, my post-doctoral research

Partner institutions: ISEP, IRSTEA (for the full time series analysis part)

Change detection is a very useful task in the field of remote sensing as it allows the study of many phenomenon: urban development analysis, global warming effect on nature assessment, aftermath of natural disasters mapping, crop rotations analysis, etc. However, this is also a very challenging task due to the following issues :

Despite the number of available remote sensing image sources, nearly all images are unlabeled, thus making it an unsupervised problem by essence. Furthermore, due to the large variety of tasks and landscapes, too few experts are available to classify and segment enough data to train a supervised neural network.
Optical images can be challenging : occlusion issues are frequent (clouds). There are usually different types of objects of various sizes and at different scales that may or may not be interesting to detect depending on the end task, these objects are described by attributes of different natures that are more or less useful to detect given types of objects (color bands, shapes, textures, coordinates, artificial indexes, etc)
Change detection with remote sensing images itself has its own challenges such as making the difference between interesting changes (constructions, crop rotation, changes in land cover, vegetation type changes) and meaningless changes due to seasonal effect or different angles from the sun light.

To solve this issue, we design a method in two steps : First detecting meaningful changes between any pair of images and optionally clustering the changes. Second, using graph synopsis, dynamic time warping and GRU-based neural networks and hierarchical clustering to sort different types of changes over long time series. The full framework is shown below.

Our "meaningful change detection" method between two images relies on a joint-autoencoder, whose principale is shown in the Figure below : It is first pre-trained to predict image t+1 from image t, and image t from image t+1, thus learning all seasonal and uninteresting changes in light, texture and vegetation. Then the auto-encoder is tested to reconstruct image t+1 from image t and vice versa and two errors maps are computed between the reconstructed images and the actual images. Due to autoencoders lack of ability to predict the future or the past, all meaningful changes show up in these error maps, thus making is possible to identify areas of interest.

In addition to being a building block for a more global method on unsupervised change classification for satellite image times series, this technique was also applied to map the damages caused by natural disasters. The figure below show an example of such application to before and after images of the 2011 Tohoku Tsunami in the Sendai region. In this work, we added an extra clustering step to distinguished the change areas between: flooded areas, destroyed and damaged buildings, and intact buildings.

Example of change detection applied to the detection of flooded areas after the Tohoku Tsunami : (a) Image before ; (b) Image after : (c) Raw reconstruction error (MSE) using joint autoencoders ; (d) Non-trivial changes area detected after Otsu thresholding. Source : Sublime et Kalinicheva 2019 [PDF]

Following changes in ARMD lesions [2019-2020]

Context: Guillaume Dupont and Clément Royer internships, part of Ekaterina Kalinicheva PhD thesis.

Partner institutions: ISEP, IHU des 15-20

ARMD (Age-Related Macular Degeneration) is the most common cause of irreversible central vision loss in older patients, and is caused by lesions that slowly grow on the retina. With the mechanisms of the disease being poorly understood, our goal was to study the progression of the lesions between patient follow-up exams. This task present the following challenges:

The OCT IR cSLO images from eye exams have a lot of contrast issues, artefacts and lighting issues, and even the textures can vary a lot from one exam to the other due to these problems.
ARMD lesions are not always easy to delineate and there is no known model explaining when they will appear or how they will expand and merge.
Due to the above point, few reliably annotated images with precisely delineated regions are available, thus making it yet another very weakly or unsupervised task.

Our first solution to this problem was to re-adapt the joint autoencoder proposed by our team for change detection between 2 remote sensing images, and see how it would work with medical images. The results are shown on the left of this text, with an example of 2 images from successive exams with a 6 month time laps, and the proposed change detection by our algorithm. Please note that while this example shows an easy case with clean and easy to delineate lesions, this is rarely as simple. It is worth mentioning that our fully unsupervised algorithm achieved good results, even with harder lesions.

Our second approach was to attempt a direct unsupervised segmentation without relying on change detection. We did so with mild results using a modified version of W-Nets. The results are shown below.

Using W-Nets to segment ARMD lesions in cSLO IR images. From left to right : original image in false color, ground-truth, proposed segmentation using W-Nets. Royer et al. 2021 [PDF]

Work on multi-view and collaborative clustering [before 2019]

Context: My PhD thesis, some of my post-doctoral work, Denis Maurel PhD thesis

Nowadays the sources of data partially describing the same elements have been multiplied: user data split on several social networks, data coming from connected objects, medical data from multiple exams or sensor, marketing data on the same clients spread on different bases, etc. This rise of huge amounts of data available from multiple sources that may contain different clusters gave birth to the field of multi-view clustering in which consists in developing clustering algorithms that can run on each view locally while accounting for what happens in the other views. In a similar way, the field of collaborative clustering can be seen as a tool for multi-view learning since it consists in developing clustering framework that enable several clustering algorithms to work together and exchange their finding to improve their local models and the subsequent partitions.

Within this context, during my PhD thesis under the supervision of Pr. Antoine Cornuéjols and Pr. Younès Bennani, I have proposed new models for collaborative clustering that make it possible for clustering algorithms of different natures to work together on a clustering task (multi-view or not). The advantages of combining different clustering algorithms are the following: different clustering algorithms can catch different types and shapes of clusters, in a multi-view context with views that feature different types of attributes it may be necessary to use different types of clustering algorithms. As part of collaboration during the PhD Thesis of Pierre Alexandre Murena, a new approach based on Kolmogorov complexity was proposed that enables an even broader spectrum of clustering algorithm to collaborate together.

The second problem I am currently tackling with collaborative and multi-view clustering is the issue of the quality of the data and to detect noisy or irrelevant views, as well as weak algorithm models that may hinder a multi-source analysis. To this end, in collaboration with Pr. Matei and Pr. Grozavu, I have among other things studied the importance of diversity and clustering stability in collaborative clustering. In the same vein, in collaboration with Pr Matei and Dr. Murena, we are currently working on a study of the theoretical properties of collaborative and multi-view method such as the stability of these multi-algorithms frameworks, the novelty that multi-view methods can bring in local partitions but also the consistency mono-algorithm models, and finally the robustness of these methods. For this work, our approach is based on Shai Ben David study of similar properties for regular clustering. Our goal is to define and find the links between stability, novelty, and consistency for multi-view and collaborative methods. This works also involves the notion of pure clustering collaborative clustering methods, and seeks to study how already developped multi-view, ensemble and collaborative clustering methods are linked and behave regarding the forementioned properties.

Finally, as part of the PhD thesis of Denis Maurel, we have been tackling two last but not less important issues regarding multi-view clustering : the issues of data privacy and missing data. While all earlier works assumed that there were no missing data and that everything could be exchanged freely between different views, it is rarely the case in practice where missing data are a huge problem. As for freely sharing the data between views, recent scandals with social networks data have shown that privacy issues are also an important problem to consider when developing Machine Learning algorithms. To this end, we have been working on reshaping most multi-view approach using deep autoencoder and other deep neural networks so that 1) the information exchanged between the views and algorithms are anonymous and heavily encrypted, and 2) missing data can be inferred or reconstructed given that there is enough information on the other views.

Google Sites

Report abuse