Vincent Divol (CREST, ENSAE)
Anna Korba (CREST, ENSAE)
Jaouad Mourtada (CREST, ENSAE)
Sessions from the 2025-26 academic year. See other tabs for sessions from previous years.
Monday, September 15, 2:00pm
Vianney Perchet (CREST, ENSAE)
Last Iterate Convergence for Uncoupled Learning in Zero-Sum Games with Bandit Feedback
In this talk, I will introduce the problem of learning in zero-sum game, and especially for the problem of "last-iterate" convergence, unlike the traditional literature that looks at the average convergence (we argue it makes more sense). The interesting property is that the optimal rate is T^{-1/4} which is quite unusual (and unexpected) in this literature.
Monday, September 29, 2:00pm
Pierre Marion (INRIA Paris)
Large Stepsizes Accelerate Gradient Descent for (Regularized) Logistic Regression
Deep learning practitioners usually use large stepsizes when training neural networks. To understand the impact of large stepsizes on training dynamics, we consider the simplified setting of gradient descent (GD) applied to logistic regression with linearly separable data, where the stepsize is so large that the loss initially oscillates. We study the training dynamics, and show convergence and acceleration compared to using stepsizes that satisfy the descent lemma. I will show some key ideas from the proof and, if time allows, discuss what happens when adding a regularization term.
Monday, October 6, 2:00pm
Raphaël Berthier (INRIA, Sorbonne Université)
Diagonal linear networks and the lasso regularization path
Diagonal linear networks are neural networks with linear activation and diagonal weight matrices. The interest in this extremely simple neural network structure is theoretical: its implicit regularization can be rigorously analyzed. In this talk, I will show that the training trajectory of diagonal linear networks is closely related to the lasso regularization path, even when no explicit sparse penalization is used. In this connection, the training time plays the role of an inverse regularization parameter. As a consequence, an earlier stopping time leads to a sparser linear model.
Monday, November 3, 2:00pm
Christophe Giraud (Université Paris Saclay)
Computational barriers in learning: high-dimensional phenomenon
In many high-dimensional problems, the best polynomial-time estimators fall short of the information-theoretic limits that are provably attainable without computational constraints. The low-degree polynomial framework has emerged as a powerful tool for analyzing the fundamental capabilities of polynomial-time algorithms. Building on our recent advances in the study of low-degree lower bounds, we will explore two notable high-dimensional phenomena:
1. Complex structures – We will show that polynomial-time algorithms may fail to leverage certain low-dimensional yet complex structures.
2. Beyond predictions from statistical physics – We will highlight cases where the computational barriers suggested by tools from statistical physics do not remain valid in specific high-dimensional regimes.
Monday, November 10, 2:00pm
Marc Jourdan (EPFL)
Advances in Pure Exploration in Bandits: Non-Asymptotic and Private
In pure exploration problems for stochastic multi-armed bandits, the goal is to answer a question about a set of unknown distributions (for example, the efficacy of a treatment) by strategically sampling from them, while providing guarantees on the returned answer. The archetypal example is the best arm identification problem, where the task is to find the arm with the largest mean. Top Two algorithms, which select the next arm to sample from among a leader and a challenger, have received significant attention in recent years due to their simplicity and interpretability.
In this talk, I will present recent advances on two complementary aspects of pure exploration: achieving non-asymptotic guarantees and ensuring differential privacy. First, we propose a Top Two algorithm which has an asymptotically optimal expected sample complexity, and also provides anytime guarantees on the probability of misidentifying a sufficiently good arm. Second, we show how the Top Two principle can be combined with differential privacy mechanisms, leading to algorithms that preserve near-optimal efficiency while ensuring privacy guarantees. These results not only deepen our theoretical understanding but also enable more practical and privacy-aware bandit algorithms.
Monday, November 24, 2:00pm
Arthur Stéphanovitch (CREST, ENSAE)
Regularity of the score and convergence rates of generative diffusion models
We show that in generative modeling with diffusion processes, the score function naturally inherits the regularity of the target distribution. This adaptive behavior provides a concise proof that these models attain minimax optimal rates for density estimation.
Monday, December 1, 2:00pm
Antonio Ocello (CREST, ENSAE)
Convergence Analysis of Diffusion Models in Wasserstein distance: a log-concave story
Generative models are attracting growing attention across various applied domains, including insurance and finance. Their potential lies in their capacity to capture and reproduce complex data patterns, crucial for realistic modeling and decision-making under uncertainty. Among the available methods, Score-Based Generative Models (SGMs), also known as diffusion models, offer a flexible framework to sample from complex, high-dimensional distributions. However, a key challenge lies in rigorously understanding their convergence.
In this talk, I will present recent advances in the theoretical analysis of SGMs. First, I will provide explicit bounds on Wasserstein-2 distance under the log-concave assumption of the target data distribution. Second, I will generalize this bound beyond the log-concave settings—such as for mixtures of Gaussians.
This talk is based on joint work with Stanislas Strasman, Claire Boyer, Sylvain Le Corff, and Vincent Lemaire (TMLR 2024 – https://openreview.net/forum?id=BlYIPa0Fx1), as well as a recent collaboration with Marta Gentiloni-Silveri (ICML 2025 – https://arxiv.org/pdf/2501.02298).
Monday, December 8, 2:00pm
Sinho Chewi (Yale University)
Discretization and distribution learning in diffusion models
First, I will review some literature on discretization of diffusion models, focusing on the use of randomized midpoints for deterministic vs. stochastic samplers. Then, I will argue that such sampling guarantees reduce distribution learning, in the form of learning to generate a sample, to score matching. To complement this result, we reduce other forms of distribution learning (parameter estimation and density estimation) to score matching as well. This leads to new consequences for diffusion models, such as asymptotic efficiency of a DDPM-based parameter estimator and algorithms for Gaussian mixture density estimation, as well as to a general approach for establishing cryptographic hardness results for score estimation.
Monday, January 5, 2:00pm
Olga Klopp (ESSEC)
Semi‑Supervised Learning on Graphs with GNNs
We study semi‑supervised node prediction on graphs where responses arise from a graph‑aware feature operator followed by a smooth regression map. Within a class combining skip‑connected GCN propagation with a fully connected ReLU network, we (i) derive an oracle inequality for population risk under random label masks that separates approximation and estimation error and exposes dependence on the labeled fraction, covering numbers, and a receptive‑field constant; (ii) show skip connections exactly represent multi‑hop polynomial filters, mitigating over‑smoothing; (iii) give covering‑number bounds; and (iv) quantify robustness of our algorithm. These results link classical graph regularization and modern GNN design.
Monday, January 19, 2:00pm
Elsa Cazelles (CNRS, IRIT)
Principal component analysis for probability distributions: a review
I will present different ways of conducting principal component analysis of datasets whose elements are probability distributions. For that purpose, I will consider the Riemannian-like structure of the space of probability distributions (with moments of order 2) endowed with the Wasserstein metric. The nice geometric properties (such as the existence of geodesics) of the Wasserstein space do not, however, allow applying classical statistical learning tools such as PCA for Hilbert spaces. Using techniques borrowed from Riemannian geometry, I will present different tools to produce a meaningful second order statistical analysis of a dataset of probability measures, focusing on the one-dimensional case, the Gaussian case, and the linearization of the Wasserstein metric.
Monday, January 26, 2:00pm
Eric Moulines (EPITA & MBZUAI)
Sampling Posteriors with Implicit Priors: Diffusion Models, Guidance Bias, and SMC
We study Bayesian sampling for inverse problems where the goal is to reconstruct a signal $X$ from noisy observations $Y=A(X)+\sigma Z$ while providing uncertainty quantification. The posterior can be written as $\pi(dx)\propto g(x)\rho(dx)$, where $g$ is a potential (typically the likelihood) and $\rho$ is an implicit prior approximated by a diffusion model (DDPM). This setting makes standard approaches that rely on evaluating the prior density intractable. We review guidance methods that modify the denoising dynamics using score-based terms (notably via Tweedie's formula), and we highlight the biases introduced by approximating intermediate potentials. We then present a "sequence of distributions" viewpoint: the exact posterior path is replaced by a chain of distributions bridging a reference Gaussian to the target, described through a Feynman–Kac representation and simulable with SMC/particle methods. Finally, we discuss Monte Carlo guidance schemes and mixture-based approaches, including a data-augmentation formulation and a deterministic Gibbs sampler, and we illustrate these ideas on imaging tasks (super-resolution, deblurring, inpainting) and audio source separation.
Monday, February 9, 2:00pm
Alexandra Carpentier (Universität Potsdam)
Statistical and computational challenges in unsupervised learning: focus on ranking
Ranking problems are prevalent in modern statistical, machine learning, and computer science literature. This includes a variety of practical situations ranging from ranking experts/workers in crowd-sourced data, ranking players in a tournament or equivalently sorting objects based on pairwise comparisons. A main challenge in this field is to construct an estimator of the rank of the experts, based on incomplete and noisy data.
In this talk, we focus on understanding the problem of ranking both from an informational - namely, characterizing the fundamental statistical thresholds for optimal estimation - and a computational - namely, also characterising the fundamental limits of computationally efficient estimation - perspective. A core question for these problems is on whether statistical optimality is compatible with computational efficiency.
To do that, we first consider the simpler sub-problem of sub-matrix detection and estimation, which is useful to apprehend the more complex problem of ranking - and we will particularly focus on computational lower bounds. Based on results for this problem, we explain how they can be used to solve the more challenging problem of ranking.
Monday, March 9, 2:00pm
Aymeric Dieuleveut (Ecole polytechnique)
Will computers replace optimization researchers? Computer-aided proofs in first-order optimization, a case study on error feedback
First-order methods are widely used in optimization and machine learning, and their behavior often analyzed through the spectrum of worst case convergence rates. Obtaining such guarantees is often difficult and both time consuming and error-prone. Starting with the work of Drori and Teboulle (2014), novel techniques have been used to gain numerical insights, leading to the release of various performance estimation (PE) software.
In this talk, I will show how various computer-aided techniques can be used to study first-order optimization methods in a systematic way. From performance estimation problems with automated Lyapunov discovery, to symbolic regression and computer algebra systems, novel tools completely reshape the way we approach theory of optimization.
As a main example, I will focus on error feedback methods used with compressed communication in distributed optimization. While error feedback has been widely studied, existing theory often provides untight (thus unreliable) bounds. I will present tight analyses with matching lower bounds that allow a fair comparison between error feedback schemes and standard compressed gradient descent, and help explain when error feedback is useful and when it is not.
Overall, the talk aims to show how various computer-aided proofs can lead to clearer and more reliable insights into first-order optimization methods.
Monday, March 16, 11:00am, room 3049
Gábor Lugosi (Universitat Pompeu Fabra)
Structure learning and property testing in graphical models
The dependence structure of high-dimensional distributions is often modeled by graphical models. The problem of learning the graph underlying such distributions has received a lot of attention in statistics and machine learning. In problems of very high dimension, it is often too costly even to store the sample covariance matrix. We propose a model in which one can query single entries of the covariance matrix. We construct efficient algorithms for structure recovery in Gaussian graphical models with query complexity that is quasi-linear in the dimension. We present algorithms that work for trees and, more generally, for graphs of small treewidth. We also discuss hypothesis testing of properties of the underlying graph. The talk is based on joint work with Sofiya Burova, Francisco Calvillo, Luc Devroye, Jakub Truszkowski, Vasiliki Velona, and Piotr Zwiernik.
Monday, March 16; Thursday, March 19; Thursday, March 26; Monday, March 30 (2:00-5:00pm, room 2005)
Taiji Suzuki (Tokyo University & RIKEN)
Feature learning perspective of deep foundation models: Statistical and optimization theories
Syllabus available here
Monday, March 23, 2:00pm
Thomas Berrett (University of Warwick)
Permutation two-sample testing under local differential privacy
Personal and sensitive data is now collected at larger scales than ever before. Growing concern from data subjects and regulatory bodies, however, has led to an increased demand for statistical procedures that do not compromise the privacy of the individuals whose data are collected and analysed. In this talk I will discuss recent work on two-sample testing under a local differential privacy constraint where a permutation procedure is used to calibrate the tests. While permutation testing is a classical resampling technique, popular due to its ease of implementation and uniform Type I error control, its use under local privacy constraints is complicated by the fact that access to the data is limited. In this work we design appropriate privacy mechanisms, both interactive and non-interactive, that allow for permutation tests. Our analysis shows that these lead to minimax optimal separation rates in both discrete and continuous settings, with interactive procedures being significantly more powerful. This is recent joint work with Alexander Kent and Yi Yu (https://arxiv.org/abs/2505.24811).
Tuesday, March 31, 2:00pm
Krishna Balasubramanian (UC Davis)
Finite-Particle Convergence of (Regularized) Stein Variational Gradient Descent
Stein Variational Gradient Descent (SVGD) is a widely used particle-based algorithm for approximate Bayesian inference. To address the intrinsic bias of SVGD, a regularized variant (R-SVGD) has been proposed that applies a resolvent-type preconditioner to the kernelized Wasserstein gradient. In this talk, I present recent results establishing explicit non-asymptotic finite-particle convergence guarantees for both SVGD and R-SVGD. The analysis is based on an entropy method for interacting particle systems, showing that the time derivative of the relative entropy between the joint law of the particles and the product target measure decomposes into a dominant negative term proportional to the number of particles times the expected squared Kernelized Stein Discrepancy (KSD²), together with a smaller positive correction. This structure yields finite-particle convergence rates of order (N^{-1/2}) in KSD and Fisher information for SVGD and R-SVGD respectively. Under mild assumptions on the kernel and the target potential, the resulting bounds grow only polynomially with the dimension. The analysis further yields Wasserstein convergence rates for both algorithms, as well as marginal convergence and long-time propagation of chaos for the interacting particle system.
Monday, April 13, 2:00pm
Hugo Lavenant (Bocconi University)
Masked diffusions: how to choose an optimal schedule and how much is gained?
We look at the computational error made in the generation phase of Masked Diffusion Models, which were recently proposed for generative modelling of discrete data. These models make approximations of conditional dependence to speed up generation of new samples at the price of some bias in the sampling distribution. We explain how these models can be simply described once you do not try to make a parallel with continuous diffusion models. From this elementary description, we study the resulting computation-vs-accuracy trade-off, providing general error bounds in relative entropy, and we investigate the gains obtained by using non-constant schedule sizes.
Monday, May 4, 2:00pm
Mathias Drton (TU Munich)
Parameter identification in linear non-Gaussian causal models under general confounding
Linear non-Gaussian causal models postulate that each random variable is a linear function of parent variables and non-Gaussian exogenous error terms. We study identification of the linear coefficients when such models contain latent variables. Our focus is on the commonly studied acyclic setting, where each model corresponds to a directed acyclic graph (DAG). For this case, prior literature has demonstrated that connections to overcomplete independent component analysis yield effective criteria to decide parameter identifiability in latent variable models. However, this connection is based on the assumption that the observed variables linearly depend on the latent variables. Departing from this assumption, we treat models that allow for arbitrary nonlinear latent confounding. Our main result is a graphical criterion that is necessary and sufficient for deciding the generic identifiability of direct causal effects. Moreover, we provide an algorithmic implementation of the criterion with a run time that is polynomial in the number of observed variables. Finally, we report on estimation heuristics based on the identification result and explore a generalization to models with feedback loops.
Monday, May 18, 2:00pm
Yannick Baraud (Université du Luxembourg)
A Robust Alternative to Least Squares in Regression
In collaboration with Guillaume Maillard, we study the estimation of a regression function under weak assumptions on the error distribution. In particular, we do not assume that the errors are i.i.d., nor that they have finite variance or exponential moments; we only require them to be independent and centered (and hence integrable). In particular, when the errors are square-integrable, they may, for instance, be heteroscedastic.
Within this statistical framework, we introduce a generic estimation method that yields estimators whose performance automatically adapts to the integrability properties of the errors. For these estimators, we establish non-asymptotic risk bounds for the L_1-loss. When the regression function belongs to a linear space and the errors are Gaussian but not necessarily i.i.d., these estimators exhibit remarkable robustness properties: they may converge at parametric rate (up to a logarithmic factor) in situations where classical least squares is not even consistent.
Nevertheless, we mainly illustrate their properties in the context of estimating a regression function under a shape constraint, such as monotonicity, unimodality, or convexity. We show that the proposed estimator is not only robust with respect to this a priori shape assumption, but also exhibits adaptation properties which are similar to those established for the least squares under the additional assumptions that the errors are i.i.d. and square-integrable.
Monday, June 15, 2:00pm
Martin Wahl (Bielefeld University)
Higher-order spectral perturbation expansions: Applications to kernel matrices and manifold learning
We study spectral concentration bounds for kernel matrices as approximation of the corresponding kernel integral operator. Results are established under weak assumptions on the data setting and the reproducing kernel relying only on Mercer's condition and a local Weyl law. This allows us to deal with key features of kernel matrices, such as large multiplicities, large effective dimension, and heavy-tailed distributions. Our results apply to infinite dimensional principal component analysis, manifold learning, and non-parametric statistics. We illustrate this via two prototypcal examples: The heat kernel on the sphere and a wavelet prior from Bayesian non-parametrics.
Monday, June 29, 2:00pm
Andrea Montanari (Stanford University)
Learning in two-layer neural networks: A statistical overview
Classical statistical models are constructed so that either the number of parameters is small with respect to the sample size, or they regularization is strong enough so that their `effective dimension' is small. From this perspective, basic techniques such as maximum likelihood are not that different from more modern ones, such as the Lasso.
In contrast, modern machine learning models are vastly miss-specified and overparametrized, and only very loosely regularized. The resulting emirical risk function has many near-global optima, which are very far from each other in parameters' space. How can we make sense of their statistical properties? I will summarize what we learnt over the years by studying a simple and yet extremely rich setting: learning in two-layer networks.