Jorge Fernandez-de-Cossio-Diaz

PSL Junior Fellow in Artificial Intelligence

Postdoctoral researcher

Statistical physics and inference for biology team
Département de Physique de l'École Normale Supérieure (ENS)

24 rue Lhomond 75005 Paris, France

Contact: jfdecd[at]icloud[dot]com

Summary

I am a PSL Junior Fellow in Artificial Intelligence, working in the group led by S. Cocco and R. Monasson, part of the Statistical physics and inference for biology team, at ENS, Paris. I am interested in machine learning, statistical mechanics and biological sequence modeling. During my PhD, I also worked on mathematical models of metabolism in cell cultures.

See my full CV.

News

2024-04-22. Updated version of our preprint on RNA switches design, with additional results and experiments: 2023.05.10.540155.

2024-02-08. Two new preprints:

2024-01-26. Our work “Accelerated Sampling with Stacked Restricted Boltzmann Machines” accepted for ICLR 2024. I'll be in Vienna in the week of May 6th-10th to present it.

2024-01-25. Invited talk at IBENS Trasversal Topics #1, ENS, Paris.

2023-12-12. Invited talk at Séminaire du LPTMS, Paris.

2023-11-07. Invited talk at Meetings on Language Processing in Humans and Models, Département d'études cognitives, ENS, Paris.

2023-10-25. New preprint: Inference and design of antibody specificity: from experiments to models and back.

2023-04-01. In April 2023 we organize (together with S. Cocco and M. Weigt) a conference covering diverse aspects of biological sequence modeling and evolution, in Cargèse. See the website for more details.

2023-05-10. Recent preprints: 2023.05.10.540155, 2022.12.06.519259 (now published at eLife).

2022-09-01. Appointed Junior Fellow in Artificial Intelligence of the Paris Sciences et Lettres University (PSL). In this role, I will continue my research on topics related to machine learning, statistical mechanics, and biological sequence modeling. I will also teach courses on machine learning applied to biology and physics.

Some selected works

For my full CV, please see this link.

Disentangling representations in Restricted Boltzmann Machines without adversaries

J. Fernandez-de-Cossio-Diaz, S. Cocco, R. Monasson. Physical Review X 13, 021003 (2023) [arXiv: 2206.11600]
A goal of unsupervised machine learning is to build representations of complex high-dimensional data, with simple relations to their properties. Such disentangled representations make easier to interpret the significant latent factors of variation in the data, as well as to generate new data with desirable features. Methods for disentangling representations often rely on an adversarial scheme, in which representations are tuned to avoid discriminators from being able to reconstruct information about the data properties (labels). Unfortunately adversarial training is generally difficult to implement in practice. Here we propose a simple, effective way of disentangling representations without any need to train adversarial discriminators, and apply our approach to Restricted Boltzmann Machines (RBM), one of the simplest representation-based generative models. Our approach relies on the introduction of adequate constraints on the weights during training, which allows us to concentrate information about labels on a small subset of latent variables. The effectiveness of the approach is illustrated with four examples: the CelebA dataset of facial images, the two-dimensional Ising model, the MNIST dataset of handwritten digits, and the taxonomy of protein families. In addition, we show how our framework allows for analytically computing the cost, in terms of log-likelihood of the data, associated to the disentanglement of their representations.

Accelerated Sampling with Stacked Restricted Boltzmann Machines

J. Fernandez-de-Cossio-Diaz*, C. Roussel*, S. Cocco, R. Monasson. ICLR (2024). (*) equal contribution.
Sampling complex distributions is an important but difficult objective in various fields, including physics, chemistry, and statistics. An improvement of standard Monte Carlo (MC) methods, intensively used in particular in the context of disordered systems, is Parallel Tempering, also called replica exchange MC, in which a sequence of MC Markov chains at decreasing temperatures are run in parallel and can swap their configurations. In this work we apply the ideas of parallel tempering in the context of restricted Boltzmann machines (RBM), a paradigm of unsupervised architectures, capable to learn complex, multimodal distributions. Inspired by Deep Tempering, an approach introduced for deep belief networks, we show how to learn on top of the first RBM a stack of nested RBMs, using the representations of a RBM as ’data’ for the next one along the stack. In our Stacked Tempering approach the hidden configurations of a machine can be exchanged with the visible configurations of the next one in the stack. Replica exchanges between the different RBMs is facilitated by the increasingly clustered representations learnt by deeper RBMs, allowing for fast transitions between the different modes of the data distribution. Analytical calculations of mixing times in a simplified theoretical setting shed light on why Stacked Tempering works, and how hyperparameters, such as the aspect ratios of the RBMs and weight regularization should be chosen. We illustrate the efficiency of the Stacked Tempering method with respect to standard and replica exchange MC on several datasets: MNIST, in-silico Lattice Proteins, and the 2D-Ising model.

Designing molecular RNA switches with Restricted Boltzmann machines

J. Fernandez-de-Cossio-Diaz*, P. Hardouin*, F.-X. Lyonnet du Moutier, A. Di Gioacchino, B. Marchand, Y. Ponty, B. Sargueil, R. Monasson, S. Cocco (2023) [bioRxiv: 2023.05.10.540155]. (*) equal contribution.
Riboswitches are structured allosteric RNA molecules that change conformation in response to a metabolite binding event, eventually triggering a regulatory response. Computational modelling of the structure of these molecules is complicated by a complex network of tertiary contacts, stabilized by the presence of their cognate metabolite. In this work, we focus on the aptamer domain of SAM-I riboswitches and show that Restricted Boltzmann machines (RBM), an unsupervised machine learning architecture, can capture intricate sequence dependencies induced by secondary and tertiary structure, as well as a switching mechanism between open and closed conformations. The RBM model is then used for the design of artificial allosteric SAM-I aptamers. To experimentally validate the functionality of the designed sequences, we resort to chemical probing (SHAPE-MaP), and develop a tailored analysis pipeline adequate for high-throughput tests of diverse homologous sequences. We probed a total of 476 RBM designed sequences in two experiments, showing between 20% and 40% divergence from any natural sequence, obtaining ~30% success rate of correctly structured aptamers that undergo a structural switch in response to SAM.

Unsupervised inference of protein fitness landscape from deep mutational scan

J. Fernandez-de-Cossio-Diaz, G. Uguzzoni, A. Pagnani. Molecular Biology and Evolution 38.1: 318-328 (2021)
The recent technological advances underlying the screening of large combinatorial libraries in high-throughput mutational scans deepen our understanding of adaptive protein evolution and boost its applications in protein design. Nevertheless, the large number of possible genotypes requires suitable computational methods for data analysis, the prediction of mutational effects, and the generation of optimized sequences. We describe a computational method that, trained on sequencing samples from multiple rounds of a screening experiment, provides a model of the genotype–fitness relationship. We tested the method on five large-scale mutational scans, yielding accurate predictions of the mutational effects on fitness. The inferred fitness landscape is robust to experimental and sampling noise and exhibits high generalization power in terms of broader sequence space exploration and higher fitness variant predictions. We investigate the role of epistasis and show that the inferred model provides structural information about the 3D contacts in the molecular fold. (This work led to a patent.)

Characterizing Steady States of Genome-Scale Metabolic Networks in Continuous Cell Cultures

J. Fernandez-de-Cossio-Diaz, K. Leon, R. Mulet. PLOS Computational Biology 13, no. 11: e1005835 (2017)
In the continuous mode of cell culture, a constant flow carrying fresh media replaces culture fluid, cells, nutrients and secreted metabolites. Here we present a model for continuous cell culture coupling intra-cellular metabolism to extracellular variables describing the state of the bioreactor, taking into account the growth capacity of the cell and the impact of toxic byproduct accumulation. We provide a method to determine the steady states of this system that is tractable for metabolic networks of arbitrary complexity. We demonstrate our approach in a toy model first, and then in a genome-scale metabolic network of the Chinese hamster ovary cell line, obtaining results that are in qualitative agreement with experimental observations. We derive a number of consequences from the model that are independent of parameter values. The ratio between cell density and dilution rate is an ideal control parameter to fix a steady state with desired metabolic properties. This conclusion is robust even in the presence of multi-stability, which is explained in our model by a negative feedback loop due to toxic byproduct accumulation. A complex landscape of steady states emerges from our simulations, including multiple metabolic switches, which also explain why cell-line and media benchmarks carried out in batch culture cannot be extrapolated to perfusion. On the other hand, we predict invariance laws between continuous cell cultures with different parameters. A practical consequence is that the chemostat is an ideal experimental model for large-scale high-density perfusion cultures, where the complex landscape of metabolic transitions is faithfully reproduced.

Current teaching activities

Other activities

Software

Collaborators