Blog
2026
Embedded Files
Stable extrapolation in high-dimensional function and operator learning
October 1, 2026Stable extrapolation in high-dimensional function and operator learning
Understanding out-of-distribution (OOD) generalization is a key open problem in modern scientific machine learning. While a variety of estimators, including neural networks, are empirically known to generalize outside the training distribution, rigorous explanations for this phenomenon remain limited. Most of the existing OOD generalization theory focuses on robustness under small distribution shifts. However, empirical evidence shows that this type of analysis is unable to explain the full extent to which generalization is possible. In a new preprint, Ben Adcock, Xuemeng Wang and I investigate this issue in the context of high-dimensional function approximation and operator learning:
https://arxiv.org/abs/2609.36709
We propose a new OOD generalization theory able to substantially widen the scope of the previous results for function approximation in high dimensions and operator learning under the assumption that the ground truth has enough regularity. Specifically, we identify classes of holomorphic functions and operators able to achieve algebraically decaying OOD generalization error even under the presence of very large distribution shifts. We focus first on polynomial estimators. Then, we obtain results for deep neural networks and neural operators using polynomial emulation techniques. The paper also includes extensive numerical examples on high-dimensional functions and solution operators of paramteric PDEs.
Example of OOD generalization for a parametric Darcy flow problem. Top row: realizations of random diffusion coefficients from different distributions (the first one used for training and the remaining three for OOD testing). Bottom row: corresponding PDE solutions. For further details, see Fig. SM1 of the paper. Numerics by Xuemeng Wang.
Finite-data error bounds for approximating the Koopman operator
September 29, 2026Finite-data error bounds for approximating the Koopman operator
Daniel Fassler, Rachel Morris, Jason Bramburger and I have just submitted a new manuscript, available on the arXiv!
https://arxiv.org/abs/2609.31817
We prove new error bounds for approximating the Koopman operator from finite data via Extended Dynamic Mode Decomposition (EDMD). The Koopman operator is a very powerful tool in data-driven discovery as it allows one to "lift" a nonlinear dynamical system to a linear, infinite-dimensional one. EDMD is a Galerkin projection-based method able to reliably approximate the Koopman operator from data on a given set of observables. In the paper, stemming from Daniel's MSc thesis, we establish a direct link between EDMD and least squares approximation theory. This allows us to (i) recover previously known Monte Carlo error rates for general dictionaries; (ii) show that for analytic observables, EDMD can achieve an exponential rate of convergence; and (iii) prove new recovery guarantees for regularized EDMD using compressed sensing theory.
If you want to know more about this project, check out Daniel's blog post about it!
Exponential convergence of EDMD with respect to the number of data points for an analytic observable; see Fig. 2 in the paper for more details. Numerics by Daniel Fassler.
Sabbatical “tour” 2025-26 🌎
August 19, 2026Sabbatical “tour” 2025-26 🌎
I am truly grateful for the academic year that has just ended. It was a special one because it included my first sabbatical. Focusing on research for one year, without teaching and with minimal service responsibilities was a refreshing experience, especially after the tenure-track years 2019-25 spent in the "academic pressure cooker”!
This was a year of travels, conferences, workshops, and, most of all, research collaborations. My sabbatical "tour" included several great institutions across North America and Europe (see this post's picture), where I am lucky to be in touch with some talentent researchers and wonderful people. They are too many to be acknowledged here, but if you're curious, you can check out the "Academic visits" section in the Events page. These visits were quite productive and a couple of them already bore fruit, such as the published papers [ZDBGT26, BBCIK26, DAB26] and the preprints [KCBKB26, KBKCB26, ABG26]. I look forward to announcing more upcoming results and new projects that stemmed from my sabbatical year. This memorable year also included a plenary lecture at YAMC 2025 in Padua, an invited talk at FoCM 2026 in Vienna, and a keynote lecture at WCCM-ECCOMAS 2026 in Munich.
Some folks told me I should print out a T-shirt with the list of dates of “Simone’s Sabbatical Tour 2025-26”. I am seriously considering it! 👕🤘
Some of the insitutions visited during my sabbatical "tour" 2025-26.
Algorithmic fundations of deep learning and new compressed sensing error bounds
July 1st, 2026Algorithmic fundations of deep learning and new compressed sensing error bounds
Summer 2026 has been very productive so far and two preprints have just appeared on arXiv!
The first paper (https://arxiv.org/abs/2606.26705) stems from a collaboration with Anastasis Kratsios, Bum Jun Kim, Greg Cousins, and Haitz Sáez de Ocáriz Borde. We propose a new framework for deep neural network approximation theory that shifts the focus from function classes to algorithmic complexity. Rather than considering classic notions of regularity (Sobolev, Besov, holomorphic, etc.), our theory takes into account which elementary operations is the function made of and how are they composed, using the language of circuit complexity.
In the second preprint (https://arxiv.org/abs/2606.26459), Avi Gupta, Ben Adcock and I propose an improved class of recovery error bounds for compressed sensing. This allows for more efficient numerical function approximation schemes in high dimensions through an optimized choice of the truncation set with rigorous error control. We apply our general analysis to the cases of weighted Wiener and anisotropic Sobolev spaces.
The classical approximation theory perspective based on function spaces (black box) vs. the algorithmic complexity lens (grey box) adopted in our new paper (see Figure 2 therein). Visualization by Anastasis Kratsios.
The stories we tell (and those we don't)
June 30, 2026The stories we tell (and those we don't)
I'd like to feature an article by Kseniya Garaschuk (U Fraser Valley) I found very inspiring, entitled "The stories we tell":
https://notes.math.ca/en/article/the-stories-we-tell/
It's about the narrative of "math geniuses" and "revolutionary thinkers" that we too often emphasize in our talks and lectures. Real science is a different thing and relies more on communities than single prodigies.
Some excerpts that stuck with me:
"In our efforts to inspire, we often construct narratives that unintentionally exclude."
"The problem is [...] the stories we tell. Not only do we highlight the prodigies, the once-in-a-generation thinkers, but we also compress careers into neat timbits of brilliance."
"[...] if our goal is not only to inspire but to invite, then we need to include different kinds of “heros” and tell their stories too. Stories not just about trailblazers and firsts, but about typical paths into the discipline and the many people who follow them."
Highly recommended reading for STEM academics, professionals, and enthusiasts.
Four mathematicians featured in the article by Garaschuk. Left to right, top to bottom: Carl Friendrich Gauss, Srinivasa Ramanujan, Sophie Germain, and Emmy Noether.
Every definable neural network can learn from finite data
May 10, 2026Every definable neural network can learn from finite data
A new preprint in collaboration with Anastasis Kratsios, Greg Cousins, Haitz Sáez de Ocáriz Borde, and Bum Jun Kim is out!
https://arxiv.org/abs/2605.07097
We show that every definable neural network can successfully perform classification and regression from a finite dataset, in the sense of Probably Approximately Correct (PAC) learnability. Definability is formalized using o-minimality, a notion borrowed from logic and model theory. Notably, virtually all the architectures and individual components considered in practice are definable in this sense (...incredible, but true!). This inlcudes MLPs, CNNs, GNNs, and transformers with fixed sequence length. A key implication of our results is that finite-sample PAC learnability should be considered a baseline rather than a differentiator between neural networks architectures, since it holds for practically every model.
The class of definable neural network considered in our theory follows the structure of a 𝔾-Neural Network. This includes MLPs, CNNs, GNNs, and transformers with fixed sequence length. For more info, see Figure 1 and Definition 2.4 in the paper. Illustration by Anastasis Kratsios.
A short tour of operator learning
March 3, 2026A short tour of operator learning
I am happy to announce a new paper just uploaded to the arXiv with Nicola Rares Franco and Nick Nelsen:
https://arxiv.org/abs/2603.00819
We provide a concise (13-page) survey of recent theoretical developments in operator learning, a fast-evolving research area at the intersection of numerical analysis, statistical learning and approximation theory. The goal is to learn operators between function (e.g., Hilbert) spaces, motivated by applications such as Partial Differential Equations (PDEs) with paramteric coefficients. In our paper, we review recently established converence rates for holomorphic operators and fundamental learnability limits for different operator classes according to the mimimax framework. We conclude by identifying promising research avenues for future work.
This "mini survey paper" is my entry point in the field and writing it with Nick and Nicola has been a great learning experience. I hope this paper will be useful to researchers who want to get started in operator learning theory!
One of the operator learning frameworks considered in the paper is the composition of an encoder, a trained multilayer perceptron, and an decoder.
From compression to depth
February 18, 2026From compression to depth
Yesterday I had the pleasure to give a talk for CodEx, an online seminar series organized by John Jasper, Emily King, Dustin Mixon, and Michael Perlmutter. My talk, entitled "From compression to depth: generative compressive sensing and deep greedy unfolding for signal reconstruction", is available on YouTube. In it, after providing a brief historical overview of the last four decades of research in sparsity, compressed sensing and deep learning for signal reconstruction, I illustrate of some of my recent research in these areas. Specifically, I feature my recent work on generative compressed sensing [Berk et a. (2022), Berk et al. (2023)] and deep greedy unfolding [Taheri, Colbrook, B. (2024), Taheri, Colbrook, B. (2025)].
I also presented a similar version of this talk in the Department of Scientific Computing Colloquium at Florida State University earlier in Ferbuary, hosted by Nick Dexter.
Google Sites
Report abuse