Lars Ruthotto (Emory University) - A Machine Learning Framework for Mean Field Games and Optimal Control

We consider the numerical solution of mean field games and optimal control problems whose state space dimension is in the tens or hundreds. In this setting, most existing numerical solvers are affected by the curse of dimensionality (CoD). To mitigate the CoD, we present a machine learning framework that combines the approximation power of neural networks with the scalability of Lagrangian PDE solvers. Specifically, we parameterize the value function with a neural network and train its weights using the objective function with additional penalties that enforce the Hamilton Jacobi Bellman equations. A key benefit of this approach is that no training data is needed, e.g., no numerical solutions to the problem need to be computed before training. We illustrate our approach and its efficacy using numerical experiments. To show the framework's generality, we consider applications such as optimal transport, deep generative modeling, mean field games for crowd motion, and multi-agent optimal control.

Andreas Habring (Uni Graz) - A Generative Variational Model for Inverse Problems in Imaging

In this talk, we are concerned with the development and analysis of a novel regularization method for inverse problems in imaging. The proposed method aims to bridge the gap between recent deep learning methods and well-established variational methods such as total variation regulariza- tion. While variational methods typically have the advantages of not requiring training data and being well understood mathematically, their practical performance is still limited in particular for highly complicated image structures such as texture. On the contrary, deep learning methods are empirically known to be well suited also for texture-like structures, but often lack an underlying mathematical foundation and often require extensive training.
The proposed method aims to combine the advantages of both types of approaches. To this aim, it employs a variational regularization functional whose architecture is inspired by the one of generative convolutional neural networks. That is, the proposed functional aims to generate the unknown via multi-layer convolutions and non-linear penalties, and penalizes an associated cost. In contrast to conventional neural-nework-based approaches, however, the convolution kernels are learned directly from the measured data, such that no training is required.
Allowing also for an infimal-convolution-based combination of the proposed functional with well-established regularization approaches, we consider regularization properties of the resulting method for inverse problems in infinite dimensions. In particular, we provide results on existence and stability for a large class of inverse problems. To highlight the practical advantage of the proposed method compared to classical variational approaches, we also provide numerical results on inpainting, denoising deblurring under noise, super-resolution and jpeg-decompression with multiple test images.

Leonard Schmitz (RWTH Aachen) - Varieties over Module Homomorphisms and their Correspondence to Free Algebras

Homomorphisms of modules are fundamental objects in algebra and applied mathematics. Recent developments in symbolic computation allow a formal verification and exploration of identities between homomorphisms. Systems of those identities define varieties similar as in classical algebraic geometry. For constant-free systems one has a Galois connection to free algebras, and morphisms of varieties can be associated to homomorphisms of factor algebras via a faithful functor. This yields a purely algebraic characterization of isomorphic varieties which can be addressed with Gröbner bases from computer algebra. The suggested formalizm finds application in theory on generalized inverses and in recent developments concerning equivalent parametrizations of stabilizing controllers.

Sebastian Neumayer (TU Berlin) - Invertible Neural Networks for Inverse Problems

In this talk, I will discuss invertible neural networks, which can be used for modelling inverse problems. If properly trained, such networks provide a way of efficiently sampling from the posterior. Starting from the theoretical analysis of the related optimization problem, we will discuss some practical implications of the established results based on numerical examples. At the end, I will outline some open issues.

Johannes Maly (KU Eichstaett-Ingolstadt) - Robust Sensing of Low-Rank Matrices with Non-Orthogonal Sparse Decomposition

We consider the problem of recovering an unknown low-rank matrix X with (possibly) non-orthogonal, effectively sparse rank-1 decomposition from incomplete and inaccurate measurements y gathered in a linear measurement process A. We propose a variational formulation that lends itself to alternating minimization and whose global minimizers provably approximate X from y up to noise level. Working with a variant of robust injectivity, we derive reconstruction guarantees for various choices of A including sub-gaussian, Gaussian rank-1, and heavy-tailed measurements. Numerical experiments support the validity of our theoretical considerations.

Kai Bergermann (TU Chemnitz) - Semi-supervised Learning for Aggregated Multilayer Graphs Using Diffuse Interface Methods and Fast Matrix Vector Products

We generalize a graph-based multiclass semi-supervised classification technique based on diffuse interface methods to multilayer graphs. Besides the treatment of various applications with an inherent multilayer structure, we present a very flexible approach that interprets high-dimensional data in a low-dimensional multilayer graph representation. Highly efficient numerical methods involving the spectral decomposition of the corresponding differential graph operators as well as fast matrix-vector products based on the nonequispaced fast Fourier transform (NFFT) enable the rapid treatment of large and high-dimensional data sets. We perform various numerical tests putting a special focus on image segmentation. In particular, we test the performance of our method on data sets with up to 10 million nodes per layer as well as up to 104 dimensions resulting in graphs with up to 52 layers. While all presented numerical experiments can be run on an average laptop computer, the linear dependence per iteration step of the runtime on the network size in all stages of our algorithm makes it scalable to even larger and higher-dimensional problems. This is joint work with Martin Stoll and Toni Volkmer.

Franz Bethke (HU Berlin) - Combining the ADMM and Active Signature Methods for the Training of Neural Networks with Nonsmooth Activation Functions

Nonsmooth activation functions are popular choices in the design of neural network architectures. However, many state of the art optimization algorithms such as the many variants of stochastic gradient descent do not account for the resulting nonsmoothness of the loss function explicitly but rather rely on the optimistic assumption that points of nondifferentiability are never encountered. Consequently, there is only very limited convergence theory accessible in this situation.
We propose a new algorithm based on the alternating direction method of multipliers (ADMM). Substituting the terms of nondifferentiability for new variables allows to take independent update steps for the "smooth" and the "nonsmooth" variables of the obtained objective function. The resulting nonsmooth subproblem is piecewise quadratic and highly structured and can thus be solved (efficiently) by a carefully tailored active signature method.
Finally, we present first numerical results for a simple network design and artificial training data.

Giuseppe Giordano (Uni Salerno) - Continuous Extensions of Selected Numerical Methods for Stochastic Differential Equations

Giovanni Pagano (Uni Salerno) - Jacobian-dependent peer methods for ordinary differential equations

Maximilian Winkler (TU Braunschweig) - Heavy Ball Acceleration of the Randomized Sparse Kaczmarz Method

The randomized sparse Kaczmarz (RSK) method is a row-action method which recovers sparse solutions of high-dimensional linear systems and is known to converge to the solution of a regularized Basis pursuit problem with a linear rate. We propose an acceleration of Heavy Ball kind. For a certain range of the momentum parameter, we are able to show linear convergence of the primal iterates. Numerical experiments indicate superiority over the non-accelerated RSK method. This is joint work with Lionel Ngoupeyou Tondji.

Michael Schmischke (TU Chemnitz) - High-Dimensional Interpretable ANOVA Approximation

In this talk we present a method for the approximation of high-dimensional functions. We assume that the function is explained well by low-order interactions which is referred to as sparsity-of-effects or the Pareto principle. The method is based upon the analysis of variance (ANOVA) decomposition. Using global sensitivity indices or Sobol indices we are able to interpret our model, i.e., rank the importance of variables and variable interactions. We present results for synthetic and real data applications.

Leila Moradi (Uni Salerno) - Adapted Numerical Methods for Oscillatory Problems

Mazen Ali (EC Nantes) - Approximation of Functions with Tensor Networks

We consider two questions regarding approximation of functions with Tensor Networks (TNs): "What are the approximation capabilities of TNs?" and "What is an appropriate model class of functions that can be approximated with TNs?"
To answer the former: we show that TNs can (near to) optimally replicate h-uniform and h-adaptive approximation, for any smoothness order of the target function. Tensor networks thus exhibit universal expressivity w.r.t. isotropic, anisotropic and mixed smoothness spaces that is comparable with more general neural networks families such as deep rectified linear unit (ReLU) networks. Put differently, TNs have the capacity to (near to) optimally approximate many function classes - without being adapted to the particular class in question.
To answer the latter: as a candidate model class we consider approximation classes of TNs and show that these are (quasi-)Banach spaces, that many types of classical smoothness spaces are continuously embedded into said approximation classes and that TN-approximation classes are themselves not embedded in any classical smoothness space.

Henrik Eisenmann (MPI MiS Leipzig) - Solving two-parameter eigenvalue problems using an alternating method

We present a new approach to compute selected eigenvalues and eigenvectors of the two-parameter eigenvalue problem. Our method requires computing generalized eigenvalue problems of the same size as the matrices of the initial two-parameter eigenvalue problem. The method is applicable for right definite problems, possibly after performing an affine transformation.