Lectures
Prof. Enrique Zuazua
Affiliation: Friedrich Alexander Universität Erlangen Nürnberg - Alexander von Humboldt Professorship, Germany
e-mail: enrique.zuazua@fau.de
Title: Control and machine learning
Abstract: In this course we shall present some recent results on the interplay between control and Machine Learning, and more precisely, Supervised Learning, Universal Approximation and Normaliying flows.
We adopt the perspective of the simultaneous or ensemble control of systems of Residual Neural Networks (ResNets). Roughly, each item to be classified corresponds to a different initial datum for the Cauchy problem of the ResNets, leading to an ensemble of solutions to be driven to the corresponding targets, associated to the labels, by means of the same control.
We present a genuinely nonlinear and constructive method, allowing to show that such an ambitious goal can be achieved, estimating the complexity of the control strategies. This property is rarely fulfilled by the classical dynamical systems in Mechanics and the very nonlinear nature of the activation function governing the ResNet dynamics plays a determinant role. It allows deforming half of the phase space while the other half remains invariant, a property that classical models in mechanics do not fulfill.
The turnpike property is also analyzed in this context, showing that a suitable choice of the cost functional used to train the ResNet leads to more stable and robust dynamics.
This lecture is inspired in joint work, among others, with Borjan Geshkovski (MIT), Domènec Ruiz-Balet (IC, London), Martin Hernandez (FAU) and Antonio Alvarez (UAM)
Furthermore, there will be a coding class by Martín Hernández Salinas (FAU). (https://dcn.nat.fau.eu/martin-hernandez/)
Prof. Lars Grüne
Affiliation: Universität Bayreuth, Germany
e-mail: lars.gruene@uni-bayreuth.de
Title: High-dimensional (optimal) feedback control with neural networks
Abstract: Optimal feedback control strategies can be computed on the basis of optimal value functions, which are solutions of Hamilton-Jacobi-Bellman partial differential equations. Similarly, stabilizing feedback controls can be computed on the basis of so-called control Lyapunov functions, which can be characterized as sub- or supersolutions of Hamilton-Jacobi-Bellman equations. These facts have triggered substantial research on solving Hamilton-Jacobi-Bellman partial differential equations with numerical methods.
Traditional numerical methods for this purpose (such as finite elements, finite differences, or semi-Lagrangian schemes) rely on the representation of the desired function on a grid of points that cover the state space. In order to maintain a certain accuracy, the number of points on this grid must in general grow exponentially with the dimension of the state space (e.g., 100 points in 1d, 10 000 points in 2d, 100 000 points in 3d, ...), which quickly leads to an infeasible number of points already in moderately high space dimensions. This phenomenon is known as the curse of dimensionality.
It is known that deep neural networks provide a way to approximate functions whose effort scales much more moderately with the dimension. However, this is not true for arbitrary functions, but only for certain classes of functions with beneficial properties. In this course we will first introduce some of these function classes. We will then consider the question which properties of (optimal) control problems lead to optimal value functions or control Lyapunov functions that fall into these classes. We will give conditions for this in terms of the dynamics and the cost function of the problem and also discuss and demonstrate suitable training algorithms.
Furthermore, there will be two coding classes by Mario Sperl (Bayreuth Universität).
(https://num.math.uni-bayreuth.de/en/team/mario-sperl/index.php)
Prof. Huyen Pham
e-mail: pham@lpsm.paris
Title: Deep (reinforcement) learning methods for stochastic control and PDEs
Abstract: Deep learning methods based on the capability approximation of neural networks and efficiency of gradient descent optimizers have shown remarkable success in the recent years years for solving high dimensional partial differential equations arising notably in stochastic optimal control. We present the different methods that have been developed in the literature relying either on deterministic or probabilistic approaches: deep Galerkin, Physics Informed NN, deep BSDE, deep backward dynamic programming, and their variants. Some convergence results will be provided, and also extensions to mean-field control in the context of large population of interacting agents.
The second part of the lecture is concerned with the resolution of stochastic control in a model-free setting, i.e. when the environment and model coefficients are unknown. This is the main purpose of reinforcement learning (RL), a classical topic in machine learning, and which has attracted an increasing interest in the stochastic analysis/control community. We shall review the basics of RL theory, and present the latest developments with a special focus on policy gradients methods in continuous time.
Furthermore, there will be a coding class by Dr. Xavier Warin (FIME Lab).