The Schedule

Program

Alex Allauzen

LAMSADE Paris-Dauphine

Lipschitz Neural Networks

Abstract: 

The Lipschitz constant of the map between the input and output space represented by a neural network has seen a growing interest in the last few years in the field of deep learning.  Indeed, numerous results have shown that neural networks with a small Lipschitz constant exhibit better generalization and higher robustness of the prediction when faced to noisy inputs or adversarial attacks. However the control of the Lipschitz constant during the training process is very challenging.

Drawing inspiration from dynamical systems, this presentation describes how to build 1-Lipschitz neural networks that are both efficient and scalable.

Erik Aurel

KTH Stockholm

 On an ensemble of constrained random Gaussian quantum states

Abstract:

Gaussian quantum states are fully characterized by their first and second moments. Important examples are thermal states in quantum statistical mechanics, and squeezed states in quantum optics. Quantum states are in general only partially characterized by their marginals, i.e. by the set of reduced density matrices. In the quantum domain it may thus be that the total state is Gaussian and pure, while all the marginals are Gaussian and mixed. Necesssary and sufficient conditions for this to be possible were determined by Eisert and co-workers in 2008.

I will present a new class of random Gaussian states constrained by some or all of their single-mode marginals. I will show that this leads to fairly compact expressions for the distribution of mode-mode correlations. As an example I will consider the totality of Hawking radiation which, according to Hawking, is emitted from an evaporating black hole, additionally assuming that the total state is pure. The constrained Gaussian ensemble then implies a probability distribution over mode-mode correlations in Hawking radiation (any pair of modes). Most of these mode-mode correlations are almost surely almost vanishingly small, but for some specific pairs of modes correlations can be relatively large. Nevertheless, the likelihood that they are entangled is still small. It hence follows that a pure Gaussian state in large dimensions can mimic a set of thermal equilibrium states as in Hawking's theory both as to their reduced density matrices, and also as to the (almost complete) absence of pairwise entanglement.

The talk is based on joint work with Lucas Hackl, Pawel Horodecki, Robert H. Jonsson and Mario Kieburg, available as [arXiv:2311.10562]

Luca Biferale

University Tor Vergata Roma

Data-driven tools for Lagrangian Turbulence. 

Abstract:

We address both problems of generation and imputation of Lagrangian trajectories in turbulent flows [1].  Lagrangian turbulence reconstruction is a challenging task across various domains, including particle tracking, astronomy, and even social sciences. Recently, the success of generative diffusion models (DMs) has opened up new ways of generating Lagrangian turbulence trajectories. These models use Markovian processes

that progressively add and remove noise scale by scale, which is naturally consistent with the multiscale nature of turbulence. In this study, we introduce  a conditional DM tailored for turbulence reconstruction tasks. The inherent stochasticity of the DM provides a probabilistic set of predictions based on known measurements. The method is first applied to address temporal gaps in Lagrangian turbulence trajectories and then to reconstruct single-particle trajectories over gaps of O(10) Kolmogorov time scales, demonstrating a smaller pointwise errors compared to Gaussian Process Regression (GPR). Crucially, our approach consistently matched key statistical benchmarks across a wide range of time scales, from the forcing time scale to the dissipation range.

[1] Synthetic Lagrangian Turbulence by Generative Diffusion Models Tianyi Li, Luca Biferale, Fabio Bonaccorso, Martino Andrea Scarpolini, Michele Buzzicotti, arXiv:2307.08529, in press on Machine Intelligence (2024)

Guido Boffetta

University Torino

Turbulence and active matter: from a single single microswimmer to the collective behavior

Abstract:

I will present numerical investigation of the behavior of microswimmers in flows. In the first part I will discuss the dynamical and statistical properties of a single microswimmer in laminar and turbulent flows. 

Subsequently, I will explore the emergenge of a turbulent-like dynamics stemming from the collective motion of dense suspensions of microswimmers.

Freddy Bouchet

ENS Paris

Rare event simulations and machine learning for predicting extreme heat waves and extremes of renewable electricity production

Abstract: 

In the climate system, extreme events or transitions between climate attractors are of primarily importance for understanding the impact of climate change. Recent extreme heat waves with huge impact, or period very low production of renewable energy in the electricity system are striking examples. However, a key challenge is the lack of data, because these events are too rare and realistic models are too complex. I will discuss new algorithms and theoretical approaches, based on rare event simulations, and machine learning for stochastic processes, which we have specifically designed for the prediction of the committor function (the probability of the extreme event to occur). To illustrate the performance of these tools, I will discuss results for the study of midlatitude extreme heat waves and the extremes of renewable energy production in relation with the resilience of the electricity system.  I will also briefly explain how the same rare event simulation and machine learning tools can be used to study rare transitions between different states of the climate system, leading to abrupt climate change. I will explain past works in this direction and current research projects.

Fabio Cecconi

CNR ROMA

Title: Causation: a conceptual framework for understanding molecular control?

Abstract:

Detection of cause-effect relationships (in brief: causation) among observables and data, at single or various scales, lies at the heart of scientific quantitative analysis and modeling. In this talk, we will discuss how causation could provide a conceptual framework to study allosteric control in molecular processes, which represents a puzzling expedient exploited by organisms to operate the "fine and remote control" of biochemical reactions at the cellular level. In general, allostery is interpreted in terms of correlation analysis and its close variant principal component analysis. However, since this kind of approach does not necessarily imply causation links among the elements of the system, its results run the risk of being biologically misinterpreted. By using as a benchmark the structure of ubiquitin, we report a critical comparison of correlation-based analysis The use of ubiquitin stems from its simple structure and recent experimental evidence of an allosteric control of its binding to target substrates. We discuss the ability of correlation, response and transfer-entropy analysis to detect the role of the protein sites involved in the allosteric mechanism of ubiquitin as deduced by experiments. To maintain the comparison as much as free from the complexity of the modeling approach and the quality of time series, we describe the fluctuations of ubiquitin native-state by the Gaussian network model which, being fully solvable, allows one to derive exact expressions of the observables of interest. Our comparison suggests that a good strategy consists in combining correlation, response and transfer entropy, such that the preliminary information extracted from correlation analysis is validated by the two other indicators to discard those (spurious) correlations not necessarily associated with true causal dependencies.

Massimo Cencini

CNR Roma

Title: Learning pursuing and evasion strategies of microswimmers in a hydrodynamic environment

Abstract:

We investigate an idealized prey-predator problem in a low Reynolds hydro-dynamic environment using reinforcement learning techniques. The problem is formalized in a game theoretic framework. Two microswimmers (the agents) — the pursuer (predator) and the evader (prey) — play the following game: the pursuer has to capture the evader in the shortest possible time and the latter to stay away from its predator as long as possible. The game terminates either upon capture (pursuer wins) or if the game duration exceeds a given time (evader wins). To accomplish its goal each agent is equipped with limited steering abilities and is capable of sensing the hydrodynamic disturbances generated by the swimming opponent, which provide only partial information on its position and direction of motion.  Such hydrodynamic disturbances also modify the motion of the microswimmers, making the environment dynamically complex. We show that learning through reinforcement both agents find nontrivial and co-evolving (with the learning process) strategies to accomplish their goals.

Peter Ditlevsen

Niels Bohr Institute, University of Copenhagen, Denmark   

Tipping points in the climate (The Day after Tomorrow or what?)

The Atlantic meridional overturning circulation (AMOC) is a major tipping element in the climate system and a future collapse would have severe impacts on the climate in the North Atlantic region. In recent years weakening in circulation has been reported, but assessments by the Intergovernmental Panel on Climate Change (IPCC), based on the Climate Model Intercomparison Project (CMIP) model simulations suggest that a full collapse is unlikely within the 21st century. Tipping to an undesired state in the climate is, however, a growing concern with increasing greenhouse gas concentrations. Predictions based on observations rely on detecting early-warning signals, primarily an increase in variance (loss of resilience) and increased autocorrelation (critical slowing down), which have recently been reported for the AMOC. Here we provide statistical significance and data-driven estimators for the time of tipping. We estimate a collapse of the AMOC to occur around mid-century under the current scenario of future emissions.

B. Dubrulle

CNRS, SPEC, University Paris-Saclay, France

Turbulence at the Kolmogorov scale

If you stir strongly enough a viscous flow, it becomes turbulent and displays vortices and coherent structures of various sizes. The typical scale for energy dissipation is called the Kolmogorov scale ηand marks the transition between the power law behavior and a steep exponential decay in the wavenumber range. Therefore, scales smaller than η contains a negligible fraction of the kinetic energy. Because of that, it is often thought that scales below η are irrelevant and that “nothing interesting is happening below η”. For a long time, it was for example thought that a direct numerical simulation of a viscous fluid is “well resolved” if

its minimal grid spacing is η. Recent theoretical and experimental progresses however suggest that many interesting phenomena do happen below η and this may impact the validity of Navier-Stokes equations (NSE) as model for the dynamics of industrial, geophysical or astrophysical fluids. This talk discusses some of these phenomena using both numerical simulations and a dedicated large turbulent experiment.

Cyril Furtlehner

INRIA Paris-Saclay

Online feature learning in terms of  spectral flow processes

Abstract:   

Regression models aim to recover a noisy signal in the form of a combination of regressors, also called features in machine learning. In the context of neural networks, these evolve according to a learning process following a gradient descent of a loss function defined by the parameters of the features and training examples.

The alignment of the population covariance feature matrix with the signal is known to play a key role in the generalization properties of the model, i.e. its ability to make predictions on unseen data during training. We present a statistical physics picture of this learning process in a simplified setting. First we revisit the ridge regression to obtain compact asymptotic expressions for train and test errors valid in the random matrix regime, rendering manifest the conditions under which efficient generalization occurs. Then, in a specific online learning setting,  we derive an autonomous dynamical system in terms of elementary degrees of freedom of the problem

namely the eigenvalue of the population matrix and the spectral power of the signal on each of these modes. This allows us in particular to determine (i) the rate of information gain by the model about the signal (ii) the evolution of the relative alignment between the population matrix and the signal in terms of the flow of its power spectrum. Additionally we see how the mean flow and its variance scale with the number of parameters in absence of inductive bias of the model.

Irene Giardina

Univ. sapienza Roma

Title:

Out-of-equilibrium response, dissipation and control in flocking systems

Abstract:

Flocking systems are known to be strongly out of equilibrium. Energy input occurs at the individual level to ensure self-propulsion, and the individual motility in turn contributes to ordering, enhancing information propagation, and strengthening collective motion. However, even beyond ordering, a crucial feature of natural aggregations is response. How, then, off-equilibrium features affect the response of the system?

In this talk, I will address this issue both from an empirical and a theoretical perspective. I will first summarize what we know from experiments on natural swarms and flocks and show that out-of-equilibrium effects are stronger in one case than in the other. Then I will consider a minimal model of flocking and investigate theoretically and numerically the response behavior. Violations of equilibrium fluctuation-dissipation relations occur both at the local and at the global level and their amount peaks at the ordering transition, exactly as for the entropy production rate.  Entropy is always produced locally. However, cooperative mechanisms close to the transition spread off--equilibrium effects on the whole system, producing an out-of-equilibrium response on the global scale. This picture reconciles nicely with what is observed in the data, providing an explanatory framework for natural systems.

Mogens H. Jensen,

Niels Bohr Institute, University of Copenhagen, Denmark   

Condensate Phase Transitions and Oscillations in Cell Dynamics

When cells are damaged or stressed they respond by oscillating protein densities as have been observed for two famous proteins/transcription factors p53 and NF-kB [1]. The oscillations have a period of 3-5 hours and appear in both healthy and sick cells. p53 is a cancer gene while NF-kB plays a role in diabetes. For p53 we show that phase transitions lead to condensates of repair proteins around damage sites which occur in an oscillating fashion thus preventing Oswald ripening. The period of oscillations provides an optimal time scale for the repair mechanism [2]. By applying an external periodic protein signal, the internal oscillation can lock to the external signal and thus controls the genes. The locking occurs when the ratio between the two frequencies is a rational number leading to Arnold tongues [1]. If tongues overlap, chaotic dynamics appear which strongly influence gene production. Our findings are in good agreement with experimental data from our collaborative groups at Harvard Medical, Beijing and Taiwan.

[1] M.L. Heltberg, S. Krishna, L.P. Kadanoff and M.H. Jensen, "A tale of two rhythms: Locked clocks and chaos in biology (Review)", Cell Systems, 12, 291-303 (2021).

[2] M.S. Heltberg, A. Lucchetti, F.-S. Hsieh, D.P.M. Nguyen, S.-h.Chen and Mogens H. Jensen, "Enhanced DNA repair through droplet formation and p53 oscillations", Cell 185, 4394–4408 (2022).

Petros Koumoutzakos

Harvard University 

AI  and Scientific Computing: There is plenty of room in the middle

Over the last thirty years we have experienced more than a billion-fold increase in hardware capabilities and a dizzying pace of acquiring and transmitting massive amounts of data. Scientific Computing and, more lately, Artificial Intelligence (AI) has been key beneficiaries of these advances. In this talk I would outline the need for bridging the decades long advances in Scientific Computing with those of AI. I will use examples from fluid mechanics to argue for forming alloys of AI and simulations for their prediction and control. I will present novel algorithms for learning the Effective Dynamics (LED) of complex systems and a fusion of multi- agent reinforcement learning and scientific computing (SciMARL) for modeling and control of turbulent flows. I will also show our recent work on Optimizing a Discrete Loss (ODIL) that outperforms popular techniques such as PINNs by several orders of magnitude. I will juxtapose successes and failures and argue that the proper fusion of scientific computing and AI expertise are essential to advance scientific frontiers.

François Landes

LISN Université Paris-Saclay

Learning representations of glassy liquids with roto-translation equivariant Graph Neural Networks

Abstract:

In this talk, I will motivate the development and use of group-equivariant Neural-Networks, which cover a broad range of applications. I will focus on the case of SE(3)-equivariant GNNs (group of roto-translation) applied to glassy systems.

After quickly introducing the task (learning a representation of a glassy liquid) and sketching out the key features of Graph Neural Networks (esp. node-equivariance), I will insist on the notions of rotation equivariance and invariance, and how to embed them in a network [Thomas 2018].

I will then present our specific architecture [Pezzicoli 2023], outline our most salient results (reduced number of parameters, improved generalization and interpretability) and open with perspectives on more recent architectures.

[Thomas 2018: Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds, Google Research]

[Pezzicoli 2023: SE (3)-equivariant graph neural networks for learning glassy liquids representations]

Aurore Loisy

IRPHE Marseille

Learning to find a source of odor in turbulence

Abstract:

How to track down a source of odor when there is no track to follow? Many insects, such as mosquitoes, are capable of reaching a source of odor from dozens of meters away. While terrestrial animals can follow odor trails on the ground, such trails do not exist in the air: any smooth aerial trail is rapidly destroyed by turbulence. As a result, insects have to navigate an odor landscape made of few randomly scattered patches of odor. Any gradient-based strategy, such as chemotaxis, is doomed to fail and one must rely on cognitive strategies: strategies that rely on memory, abstract representations, and anticipation. In this talk we will show how theoretical and practical tools from artificial intelligence can help us approach and solve this problem.

What Thermodynamic Uncertainty Relations Reveal (and Don't) About the Physical World

Stéphane Mallat

Collège de France

The architexture of complexity: from image generation to the renormalisation group

Abstract

In a foundational article in 1962, Herbert Simons observes that complex systems have an architecture which relies on a hierarchic organisation. Neural networks now provide impressive generative models of many such systems. In what sense is it related ? Diffusions by denoising score matching generate impressive images. Are they generalising or memorising ? How can deep network estimations of scores avoid the curse of dimensionality ? I will relate these questions  to the renormalisation group in physics, and to models of interaction energies across scales.

Complex systems involve multiple hierarchies. Similarly, we introduce non-local interaction energies across scales, with iterated wavelet transforms. It is applied to generative models of high-dimensional systems such as turbulences or cosmological distributions of dark matter.

Bernhard Mehlig

Chalmers University Gotenborg

Using dynamical-systems methods to understand how neural networks learn

How do neural networks learn? This can be analysed and understood, in part, using concepts from dynamical-systems theory: Lyapunov exponents and coherent structures. 

For deep neural networks, the maximal finite-time Lyapunov exponent forms geometrical structures in input space, akin to coherent structures in dynamical systems. Ridges of large positive exponents divide input space into different regions that the network associates with different classes in a classification task. The ridges visualise the geometry that deep networks construct in input space, and help to quantify how the learning depends on the network depth and upon the width of the neural network [1]. 

I also explain how to use the maximal Lyapunov exponent to determine optimal parameter choices for reservoir computing. Reservoir computers are recurrent neural networks driven by a time series.  For time-series prediction, the maximal Lyapunov exponent of these driven discrete dynamical systems must be negative. I discuss which other conditions are necessary for successful time-series prediction [2].

[1] Storm, Linander, Bec, Gustavsson & Mehlig, Finite-time Lyapunov exponents of deep neural networks, Phys. Rev. Lett. 132 (2024) 057301

[2] Storm, Gustavsson & Mehlig, Constraints on parameter choices for successful time-series prediction with echo-state networks, MLST 3 (2022) 045021

[3] Bernhard Mehlig, Machine Learning with neural networks, Cambridge University Press (2021)

Marc Mézard

Bocconi University, Milano

JSTAT Lecture:

Structured Data and the Statistical Physics of Generative Diffusion

Generative models, in which one trains an algorithm to generate samples ‘similar’ to those of a data base, is a major new direction developed in machine learning in the recent years. In particular, generative models based on diffusion equations have become the state of the art, notably for image generation. These models thus provide a choice method for understanding aspects of the structure of data used in machine learning.

After an introduction to this topic, the talk will focus on the behavior of generative diffusion in the high-dimensional limit, where data are formed by a very large number of variables.  Using methods from statistical physics, we explain the various dynamical regimes that occur during the generation.

Remi Monasson

ENS Paris

Task learning through stimulation-induced plasticity

Synaptic plasticity dynamically shapes the connectivity of neural systems and is key to learning processes in the brain. To what extent plasticity mechanisms, which are intrinsically unsupervised, can be exploited to make a neural network achieve any computational task remains unclear. This question can be formulated as a machine-learning problem with constrained weight updating. In this context, we present a self-contained procedure which, through appropriate spatio-temporal stimulation control of the neurons, is able to drive rate-based neural networks with arbitrary initial connectivity towards a desired functional state. We illustrate our approach on two different computational tasks: a non-linear association between multiple input stimulations and  activity patterns (representing digit images), and the construction of a continuous attractor encoding a collective variable in a neural population. Our work thus provides a proof of principle for emerging paradigms of machine-learning based in vitro biological computation. (In collaboration with F. Borra, S. Cocco)

Giorgio Parisi

University La Sapienza, Accademia dei Lincei

Multiple Equilibria

Andrea Puglisi

CNR Roma

What Thermodynamic Uncertainty Relations Reveal (and Don't) About the Physical World

I will start with a summary of the Thermodynamic Uncertainty Relations (TUR). TURs are a series of recent results in stochastic thermodynamics, resulting in a general bound - based upon thermodynamic information (e.g. entropy production rate) - for the precision rate of any non-equilibrium current in the system. I will then discuss two applications of the TURs. First, I will review a recent experiment where we have measured the precision rate of the beating phase of bull sperms cell at different energy consumption rates. Upon such an observation we have conjectured the validity of a TUR where the bound is scaled by the number of molecular motors inside the flagellum axoneme. Arguments in favour of this conjecture come from the re-analysis of previous experiments with beating flagella, some simple scaling reasoning,  the numerical study of a new model for sperm beating and the analysis of some toy model. All these arguments points to the fact that the validity of this new TUR rests upon a strong coupling among adjacent motors in the flagellum, which is therefore a biophysical condition which we conjecture for axonemes. The second application is a theoretical bound for the mean squared displacement of a tracer particle in the presence of baths with multiple time scales and multiple temperatures, even in the presence of some kind of non-linear friction. Such a bound is seen to hold in experiments with a rotating blade immersed in a vibrofluidized granular gas. Within the second application I will discuss the general relevance of the TUR as a tool to distinguish equilibrium from non-equilibrium when only partial information of a system (e.g. a coarse-grained variable) is available.

Pierfrancesco Urbani

CEA Paris-Saclay

Dynamical mean field theory of learning algorithms

This talk focuses on two sets of questions:

    * how to characterize and benchmark the performances of stochastic gradient descent (SGD) when used to train artificial neural networks? How does the algorithm explore the loss landscape? Is the SGD noise beneficial for optimization and to what extent?

    * How to train recurrent neural networks (RNN) to perform simple tasks? How do training algorithms explore the space of synaptic weights? How are dynamical attractors approached?

I will address both sets of questions by introducing two classes of simple models which can be analyzed in the thermodynamic limit via dynamical mean field theory. In particular I will discuss a simple yet paradigmatic non-convex teacher-student optimization problem where I will show that it is possible to quantify in a precise way the nature and the beneficial role of the out-of-equilibrium noise of SGD. Moreover I will discuss a set of high-dimensional chaotic systems as simplified models of RNN and show that we can train them to perform simple tasks.

Massimiliano Viale

CNR Roma

Title: Inferring Non-Equilibrium from Incomplete Information: Theory & Practice

Summary:

We discuss the problem of inferring the thermodynamic state of a system from partial observations. For a Gaussian process, we prove that it is impossible to distinguish an equilibrium system from an out-of-equilibrium one by observing a single variable unless a response experiment is performed. We show a possible way out when it is not possible to appropriately perturb the system to get the response but several observations recorded under different experimental conditions are available. We extend the discussion to Markov processes whose evolution is ruled by Langevin equations driven by a mixture of Gaussian and Poissonian white noises. These noises necessarily break the time-reversal symmetry, leading the system to non-equilibrium steady states for which the entropy production may be different from zero even for vanishing currents. We show that a direct measurement of entropy production is typically impractical but it is simpler and more robust to detect the time-reversal symmetry breaking by looking at higher order correlations and we provide analytical expressions in some cases. Finally we put it all together in order to analyze a real experiment on granular gases.

Lenka Zdeborova

EPFL Lausanne

Title: The Backtracking Dynamical Cavity Method

Abstract:  The cavity method is one of the cornerstones of the statistical physics of disordered systems such as spin glasses and other complex systems. It can analytically and asymptotically exactly describe the equilibrium properties of a broad range of models. Exact solutions for dynamical, out-of-equilibrium properties of disordered systems are traditionally much more challenging to obtain. Even very basic questions, such as the limiting energy of a fast quench, are so far open. The dynamical cavity method partly fills this gap by considering full trajectories as variables and leveraging the static cavity method. I will introduce the backtracking dynamical cavity method that, instead of analysing the trajectory forward from initialization, analyses trajectories that are found by tracking them backwards from attractors. I illustrate that this rather elementary twist on the dynamical cavity method leads to new insight into some of the very basic questions about the dynamics of complex disordered systems.