Abstract:  Causal operators (CO), such as various solution operators to stochastic differential equations, play a central role in contemporary stochastic analysis; however, there is still no canonical framework for designing Deep Learning (DL) models capable of approximating COs. This paper proposes a "geometry-aware'" solution to this open problem by introducing a DL model-design framework that takes suitable infinite-dimensional linear metric spaces as inputs and returns a universal sequential DL model adapted to these linear geometries. We call these models Causal Neural Operators (CNOs). Our main result states that the models produced by our framework can uniformly approximate on compact sets and across arbitrarily finite-time horizons Hölder or smooth trace class operators, which causally map sequences between given linear metric spaces. Our analysis uncovers new quantitative relationships on the latent state-space dimension of CNOs which even have new implications for (classical) finite-dimensional Recurrent Neural Networks (RNNs). We find that a linear increase of the CNO's (or RNN's) latent parameter space's dimension and of its width, and a logarithmic increase of its depth imply an exponential increase in the number of time steps for which its approximation remains valid. A direct consequence of our analysis shows that RNNs can approximate causal functions using exponentially fewer parameters than ReLU networks.

This is based on joint work with Luca Galimberti and Anastasis Kratsios.

Bio: Giulia Livieri is an Assistant Professor at the London School of Economics and Political Science (LSE) since November 2022. She was a fixed-term Assistant Professor at Scuola Normale Superiore (SNS) till November 2022. Previously Giulia was a Post-doc researcher at SNS, where she obtained a Ph.D in Financial Mathematics in October 2017 with the score of 70/70 with laude. In 2013, she did a post-graduate course in Mathematical Finance at the University of Bologna where she obtained a score of 30/30 with laude and an internship at Mediobanca, a leading investment bank in Italy. Giulia graduated in 2012 in Mathematics at the University of Padova with the score of 100/100.  Her research focuses mainly on financial econometrics for the modeling of financial markets (both at high and low frequency) and Mean-Field Game (MFG). Giulia is currently developing machine learning techniques for the standard memory, forecasting, and filtered problems that appear in the parametric stochastic time series context. Also, she aims to provide a mathematical foundation based on the theory of MFGs to implement Deep Neural Network (DNN) models.

Meeting Recording: https://ucsb.zoom.us/rec/share/6aVYdnIQQIJXmteQPuQq3I4fzcHKQS-IIyY-zXN8NE5Eg36YIWoFMKkvJxi0SmS7.6W6bVQSzOvDnHpNM

Access Passcode: $e5.AyX1