Embedded Files
Mathematical Foundations of Data Science: What a wild ride!
June 27, 2025Mathematical Foundations of Data Science: What a wild ride!
The CRM thematic program Mathematical Foundations of Data Science has officially ended today. What a wild ride has been... and what an incredible success!
The initial idea of the program came in 2021 from Tim Hoheisel (McGill) who initially wanted to co-organize a workshops on optimization and data science with the CRM. We decided to go bigger and instead submitted a proposal for a whole thematic program, which was enthusiastically approved and generously supported by the CRM.
The program included four core CRM workshops, two Aisenstadt Chair lectures, one spring school, three IVADO workshops and a satellite workshop in Ottawa. These activities featured a wide array of relevant and timely topics in the modern data science landscape, falling within the broad categories of machine learning, optimization, approximation theory, numerical analysis, probability, statistics... and beyond!
We were lucky to have a crowd of high-caliber, world-class researchers participating in the various activities of the program. They are too many to list here, but I want to mention some key people: two Aisenstadt Chairs (Francis Bach and Katya Scheinberg), ten CRM-Simons scholars in residence (Ben Adcock, Gérard Ben Arous, James V. Burke, Maia Fraser, Michael Friedlander, Diane Guignard, Tias Guns, Mark Iwen, Zhenyu Liao, and Carola-Bibiane Schönlieb), and the spring school instructors (Claire Boyer, Nathan Doumèche, and Andre Milzarek). What made the program special was the presence, work and energy of all these people, together with many other excellent speakers and participants at all career levels, from established and early career scientists to talented and enthusiastic students.
I could spend many words on the incredibly strong scientific contributions of the program, but here I would like to emphasize once more the aspect that made all these years of work worth it: the people. Seeing senior and junior participants interacting during Q&As and coffee breaks, chatting with old and new colleagues over lunch, having a good time in front of a pint with program participants in the evening... these are special memories that I won't forget easily.
I'd like to conclude this post by thanking Virginie Leduc and Octav Cornea from the CRM for the incredible support (both logistical and moral) and the members of the organizing committee (Ben Adcock, Margarida Carvalho, Michael Friedlander, Diane Guignard, Mark Iwen, Adam Oberman, Courtney Paquette, Elliot Paquette, Utsav Sadana and Carola-Bibiane Schönlieb) for their enthusiasm and dedication. Last but not least, I'd like to thank Tim Hoheisel, co-chair of the program, for being an ideal partner, team player and friend, enabling the great success that this program turned out to be. Ad maiora!
CRM Thematic Program "Mathematical Foundations of Data Science". Top: word cloud summarizing the main themes featured in the thematic program. Bottom: Tim Hoheisel (McGill) and me enjoying the program we co-chaired during a wine & cheese.
CMS Summer Meeting and Daniel's defense
June 23, 2025CMS Summer Meeting and Daniel's defense
June has been a very eventful month! Beside the CRM Thematic Program "Mathematical Foundations of Data Science" still ongoing (to which I will dedicate a future post), June has been characterized by memorable research events and achievements.
Ben Adcock (SFU), Giuseppe Alessio D'Inverno (SISSA) and I have co-organized a new edition of the scientific session "Mathematics of Machine Learning" at the 2025 CMS Summer Meeting in Québec City. This was the session's eight edition (!), part of a project spearheaded by Ben Adcock and me in Winter 2021. The session has enabled many connections and research collaborations among Canadian researchers and beyond since then. This Summer, the themes presented included physics-informed machine learning, surrogate modelling, deep unfolding, statistical learning theory and fairness in machine learning. From my research group, Sina M.-Taheri (Concordia) talked about his PhD research on deep algorithm unfolding and Giuseppe Alessio D'Inverno (SISSA) presented our recent work with Kylian Ajavon (Concordia) on sparse polynomial surrogates for diffusion on graphs.
At the same meeting, Rahul Padmanabhan (Concordia) presented a poster on his Master's thesis on deep nets and transformers for matrix function approximation and received a CMS President's Award. Well done, Rahul!
Last but not least, today Daniel Fassler (Concordia), co-supervised with Jason Bramburger, just defended his Master's thesis on finite data error bounds for extended dynamic mode decomposition. Daniel will start his PhD studies at Concordia in the Fall. Congrats, Daniel!
CMS Summer Meeting 2025. Top: the speakers and some participants of the session "Mathematics of Machine Learning". Bottom: Rahul in front of his poster, winner of the CMS President's Award.
New preprint! Deep Greedy Unfolding
May 23, 2025New preprint! Deep Greedy Unfolding
Motivated by the need for enhancing the interpretability of deep learning, in our new preprint Sina M.-Taheri, M. Colbrook and I propose a new neural network unrolling framework for greedy sparse recovery algorithms:
https://arxiv.org/abs/2505.15661
The main challenge is the reliance of greedy algorithms on the non-differentiable argsort operator, which prevents the use of gradient-based approaches for training. To address this, we introduce “soft” differentiable versions of Orthogonal Matching Pursuit (OMP) and Iterative Hard Thresholding (IHT). Our analysis admits asymptotic convergence and non-asymptotic recovery guarantees. Moreover, we demonstrate trainability of the proposed networks in conjunction with weighted sparse recovery, achieving significant performance gains for compressed sensing—especially in the heavily undersampled regime.
Our framework takes meaningful strides toward more interpretable and trustworthy AI and speaks to both theoreticians and practitioners in the fields of sparse recovery and deep learning, two major pillars of contemporary data science. While OMP and IHT have been extensively studied in the literature, unlike prior approaches that rely on heuristic modifications, our contribution offers a novel mathematical treatment of their algorithmic differentiability for their integration into gradient-based optimization.
The benefits of deep greedy unfolding. OMP-Net, one of the deep greedy unfolded neural network architectures proposed in our paper, can outperform standard OMP for compressed sensing reconstruction in the undersampled regime. For more details, see Figure 5.2 in the paper. Numerics by S. M.-Taheri.
Thematic program: Mathematical Foundations of Data Science
May 6, 2025Thematic program: Mathematical Foundations of Data Science
I am excited about the start of the thematic program Mathematical Foundations of Data Science, happening at the Centre de Recherches Mathématiques (CRM) over the next two months. The program consists of various activities including workshops, a spring school, distinguished (Aisenstadt) lectures, and long-term Simons visitors.
The thematic program's activities focus on (but are not restricted to) optimization, high-dimensional probability, approximation theory, scientific machine learning, inverse problems, and their applications in data science.
The idea of this program was born in 2021--a couple of lifetimes ago--with Tim Hoheisel and it's great to finally see this event happening!
For more information on the program activities, see
Deep Learning Approximation of Matrix Functions: Congrats, Rahul!
April 3, 2025Deep Learning Approximation of Matrix Functions: Congrats, Rahul!
Rahul Padmanabhan, a M.Sc. student in my group, just defended his thesis. In it, Rahul investigates the use of Deep Neural Networks (DNNs) and transformers for approximating matrix functions, ubiquitous in scientific applications, from continuous-time Markov chains (matrix exponential) to the stability analysis of dynamical systems (matrix sign function).
Rahul's work makes two main contributions. First, he proves theoretical bounds on the depth and width requirements for ReLU DNNs to approximate the matrix exponential. Second, he uses transformers to approximate general matrix functions and compare their performance to feedforward DNNs. Through extensive numerical experiments, he demonstrates that the choice of matrix encoding scheme significantly impacts transformers' performance. His results show high accuracy in approximating the matrix sign function, suggesting transformers' potential for advanced mathematical computations.
Rahul's thesis will soon be available on Concordia's Spectrum Research Repository.
Matrix function approximation via transformers. We approximate some popular matrix functions (see table's rows) via transformers based on different encodings (see table's columns). Numbers in the table indicate the success rate over 10,000 random 3 x 3 matrices. An experiment is considered successful if it returns an approximation with l1-relative error below 0.05. Transformers are particularly effective for the approximation of the matrix sign function. Numerics by Rahul Padmanabhan.
BIRS workshop "Uncertainty Quantification in Neural Network Models"
February 22, 2025BIRS workshop "Uncertainty Quantification in Neural Network Models"
This week I attended the Banff International Research Station (BIRS) workshop Uncertainty Quantification in Neural Network Models in the gorgeous Banff, AB, Canada. The workshop featured engaging lectures, tutorials, breakout discussions and poster presentations on a diverse range of topics in the areas of uncertainty quantification and neural networks. They included statistical learning, approximation theory, Bayesian methods, manifold learning, and surrogate modelling.
I presented a poster entitled Practical existence theorems for deep learning approximation in high dimensions, surveying my recent work in the area in collaboration with postdocs, students, and researchers across Canada, the U.S. and Europe. This poster is a good entry point if you want to get started on the topic of practical existence theory. To see the full PDF version of the poster, click here.
I also took the opportunity to add a new page on the research section of this website collecting posters presented by me and my research team over the past few years. Check it out!
Top: Part of the poster I presented a the BIRS workshop (click on the image to see its full PDF version). Bottom: Workshop participants and Cascade mountain in the background.
Reconstruction of frequency-localized functions via least squares and deep learning
February 17, 2025Reconstruction of frequency-localized functions via least squares and deep learning
Frequency-localized and bandlimited functions play a crucial role in signal processing and its applications. In our new paper, Martina Neuman (U Vienna), Andres F. Lerma Pineda (U Vienna), Jason Bramburger (Concordia), and I study reconstruction methods for these functions from pointwise samples via least squares and deep learning.
https://arxiv.org/abs/2502.09794
Least squares approximation based on Slepian (or Prolate Spheroidal Wave Function) basis expansions is a classical technique for the recovery of frequency-localized signals. Yet, explicit recovery guarantees are not available in the setting of random pointwise data. Our paper addresses this gap by providing rigorous recovery theorems based on polynomial approximation theory. Building on our least squares analysis, we also prove a new practical existence theorem for deep learning. This result shows the existence of a partially trained neural network that can accurately approximate frequency-localized functions from pointwise data. We also carry out a numerical comparison between least squares and deep learning on simple one- and two-dimensional frequency-localized functions.
The Slepian basis. Odd and even elements of the Slepian (or Prolate Spheroidal Wave Function) basis over the interval [-1,1]. This basis has useful space-frequency localization properties that are central in our analysis. For further details, see Figure 1 of the paper. (Plots by A.F. Lerma Pineda.)
Surrogate models for diffusion on graphs via sparse polynomials
February 12, 2025Surrogate models for diffusion on graphs via sparse polynomials
Diffusion equations on graphs are an important family of mathematical models in computational and applied mathematics. They are utilized in network science applications including social networks, marketing, traffic flows, and epidemic modelling. In a new preprint, G.A. D'Inverno (SISSA), K. Ajavon (Concordia) and I propose polynomial surrogate methods based on least squares and compressed sensing for diffusion on graphs, focusing on networks with community structure.
https://arxiv.org/abs/2502.06595
From the practical perspective, we implement this methodology and show its success on synthetic and real-world graphs. From the theoretical perspective, we show that the corresponding solution map is holomorphic and provide corresponding convergence rates for least squares and compressed sensing-based surrogate models.
Twitter Interaction Network for the US Congress used in our numerical experiments. This network represents the Twitter interaction network for the 117th United States Congress, both House of Representatives and Senate. For more details, see Figure 10 of the paper. (Visualization by G.A. D'Inverno.)
Google Sites
Report abuse