Program 

Abstract: The application of DNNs to the denoising problem has resulted in highly performant denoisers for a wide range of applications from photographic image restoration to medical imaging. However, these images are typically subjected to relatively low degree of noise compared to applications such as cryogenic electron microscopy (cryo-EM), where noise power dominates the clean signal to the point where traditional denoising methods fail. In this talk, we will study the related problem of multireference alignment (MRA). Here, a single clean signal is observed at random shifts and subjected to additive noise and the goal is to recover the original signal. We should how a transformer architecture can be used to encode a signal prior that is used to aggregate the information from the entire set of observations to yield a superior denoising method.

Abstract: We start with a basic introduction on deep learning approaches for inverse problems. We then discuss several network architectures and their potential to be used as regularization schemes for inverse problems. In particular we investigate deep prior networks. Using the LISTA architecture in a deep prior network allows to proof equivalences to classical regularization schemes.  On the experimental side we first  low dose CT reconstructions.

The second part of the talk is devoted to neural network concepts for solving PDE based inverse problems.
We discuss several network cocnetps for solving forward PDE problems and their extension to parameter identification problems.We close the talk with a PDE based industrial application.

This is joint work with Daniel Otero Baguer, Johannes Leuschner, Maximilian Schmidt, Sören Dittmer, Daniel Otero Baguer, Derick Nganyu, Jianfeng Ning.

Abstract: We study utilising a deep learning-based framework in the Bayesian inverse problem of photoacoustic tomography. The approach is based on the variational autoencoder (VAE) and a recently proposed uncertainty quantification variational autoencoder (UQ-VAE). In the VAE and UQ-VAE, an approximation of ‘the true’ posterior distribution is estimated by minimising a divergence between the true and estimated posterior distributions using a neural network. Results of numerical simulations are shown. This is joint work with Teemu Sahlström, University of Eastern Finland. 

Abstract: Data-driven solution techniques for inverse problems, typically based on specific learning strategies, exhibit remarkable performance in applications like image reconstruction tasks. These learning-based reconstruction strategies often follow a two-step scheme. First, one uses a given dataset to train the reconstruction scheme, which one often parametrizes via a neural network. Second, the reconstruction scheme is applied to a new measurement to obtain a reconstruction. We follow these steps but specifically parametrize the reconstruction scheme with invertible residual networks (iResNets). We demonstrate that the invertibility opens the door to investigations into the influence of the training and the architecture on the resulting reconstruction scheme. In particular, these investigations reveal a formal link to the regularization theory of linear inverse problems. In this context we investigate the effect of different iResNet architectures, loss functions, and prior distributions on the trained network.  Moreover, we analytically optimize the parameters of specific classes of architectures in the context of Bayesian inversion, showing the influence of the prior and noise distribution on the solution.

Abstract: Starting from an example of data-driven CT-reconstruction, we investigate learned spectral regularizers as a simple data-driven approach for the regularization of inverse problems. We show that the optimal solution with respect to the expected squared L2-error is a convergent regularization method in the stochastical formulation. However, inspection of the range condition reveals an oversmoothing effect of the optimal regularizer. Further, we will subsume our approach in the more general framework of learning optimal Tikhonov regularizers.

Abstract: TBA

Abstract: TBA

Abstract: TBA

Abstract: This talk introduces two connections between operator learning and inverse problems. The first involves framing the supervised learning of a linear operator between function spaces as a Bayesian inverse problem. The resulting analysis of this inverse problem establishes posterior contraction rates and generalization error bounds in the large data limit. These results provide practical insights on how to reduce sample complexity. The second connection is about solving inverse problems with operator learning. This work focuses on the inverse problem of electrical impedance tomography (EIT). A new type of neural operator is trained to directly map the data (the Neumann-to-Dirichlet boundary map, a linear operator) to the unknown parameter of the inverse problem (the conductivity, a function). Theory based on emulating the D-bar method for direct EIT shows that the EIT solution map is well-approximated by the proposed architecture. Numerical evidence supports the findings in both settings.

Abstract: We discuss deep neural networks for Bayesian inference and uncertainty quantification in inverse problems governed by nonlinear dynamical systems. In particular, we will consider the parameter estimation for a Hodgkin-Huxley model. We employ neural networks to approximate reconstruction maps for model parameter estimation from observational data. The observational data comes from the solution of the underlying ODE and takes the form of a time series representing a dynamically spiking membrane potential of a biological neuron. We consider this application since it is governed by a simple dynamical system that poses significant mathematical and computational challenges for parameter inference. We study prediction errors as a function of input data, network architecture, and noise perturbations. Our results demonstrate that deep neural networks enable us to estimate the parameters of the dynamical system as well as noise parameters in an efficient way. This is a work in progress, and several non-trivial challenges remain.This is joint work with Johann Rudi (Virginia Tech) and German Villalobos (University of Houston).

Abstract: We present a data-driven regularization method for inverse problems that generalizes [1]. Our approach consists of two steps. In the first step, an intermediate reconstruction is performed by applying a regularised inverse such as truncated singular value decomposition (SVD).  In the second step, a trained deep neural network is used to recover the surpressed coefficients. We show that the proposed scheme provides a convergent regularisation method. 

Abstract: TBA

Abstract: TBA

Abstract: Quite often,  coefficient identification in PDEs from boundary observation can be written as the  concatenation of a linear ill-posed with a nonlinear well-posend problem. While this fact served as our original motivation for studying structure exploiting regularization methods under a so-called range invariance condition on the forward operator, it has turned out that the same framework seems to apply to linear inverse problems in which the unknown is represented by a neural network.

Abstract: Wasserstein gradient flows of maximum mean discrepancy (MMD) functionals with non-smooth Riesz kernels show a rich structure. For example, following the flow in two dimensions, empirical measures can become absolutely continuous ones and conversely. 

We propose to approximate the backward scheme of Jordan, Kinderlehrer and Otto for computing such Wasserstein gradient flows as well as a forward scheme for so-called Wasserstein steepest descent flows by generative neural networks (NNs). Since we cannot restrict ourselves to absolutely continuous measures, we have to deal with transport plans and velocity plans instead of usual transport maps and velocity fields.

Indeed, we approximate the disintegration of both plans by NNs which are learned with respect to the appropriate loss functions. In order to evaluate the quality of both neural schemes, we benchmark them on the interaction energy. Here we provide analytic formulas for Wasserstein schemes starting at a Dirac measure and show their convergence as the time step size tends to zero. 

In the second part of the talk, we present results on generative sliced MMD flows with Riesz kernels.

This is joint work with Johannes Hertrich, Fabian Altekrüger, Paul Hagemann and Christian Wald (TU Berlin).

Abstract: Inverse problems are mathematically ill-posed. Thus, given some (noisy) data, there is more than one solution that fits the data. In recent years, deep neural techniques that find the most appropriate solution, in the sense that it contains a-priori information, were developed. However, they suffer from several shortcomings. First, most techniques cannot guarantee that the solution fits the data at inference. Second, while the derivation of the techniques is inspired by the existence of a valid scalar regularization function, such techniques do not in practice rely on such a function, and therefore veer away from classical variational techniques. In this work we introduce a new family of neural regularizers  for the solution of inverse problems. These regularizers are based on a variational formulation and are guaranteed to fit the data.

We demonstrate their use on a number of highly ill-posed problems, from image deblurring to limited angle tomography.

Abstract: In the context of parameter dependent PDEs, Deep Learning-based Reduced Order Models (DL-ROMs) are a class of model surrogates that can efficiently approximate the parameter-to-solution map, providing an appealing alternative to classical approaches, such as the Reduced Basis method, especially when tackling nonlinearities or a slow decay in the Kolmogorov n-width. Once trained, DL-ROMs are extremely efficient, providing thousands of simulations in fractions of a second, which makes them particularly suited for many-query applications, such as those typical of Uncertainty Quantification and inverse problems.

Here, focusing on the case of PDEs with (unbounded) stochastic coefficients, we discuss the theoretical properties of DL-ROMs and their usage for solving forward and inverse problems. In particular, we start by providing practical insights on the choice of the latent dimension of deep autoencoders, which we employ for encoding and decoding a high-fidelity discretization of the solution manifold, thus extending the results in [1] to the stochastic scenario. Then, we discuss the importance of using specific architectures, such as convolutional layers or mesh-informed architectures, in the design process of deep autoencoders. To this end, we derive suitable error bounds that can guide domain practitionares in the design of efficient DL-ROMs. Finally, we present some numerical results concerning the application of DL-ROMs, and similar techniques, to forward and inverse problems, ranging from fluid dynamics to biomedical engineering.

References

[1] Franco, N. R., Manzoni, A., and Zunino, P. (2022). A Deep Learning approach to Reduced Order Modelling of Parameter Dependent Partial Differential Equations. Mathematics of Computation.

[2] Franco, N. R., Fresca, S., Manzoni, A., and Zunino, P. (2023). Approximation bounds for convolutional neural networks in operator learning. Neural Networks.

[3] Franco, N. R., Manzoni, A., and Zunino, P. (2022). Mesh-Informed Neural Networks for Operator Learning in Finite Element Spaces. arXiv preprint.

[4] Fresca, S., Dede, L., and Manzoni, A. (2021). A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized PDEs. Journal of Scientific Computing.

[5] Fresca, S., and Manzoni, A. (2022). POD-DL-ROM: enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decomposition. Computer Methods in Applied Mechanics and Engineering.

[6] Brivio, S., Fresca, S., Franco, N. R., and Manzoni, A. (2023). Error estimates for POD-DL-ROMs: a deep learning framework for reduced order modeling of nonlinear parametrized PDEs enhanced by proper orthogonal decomposition. arXiv preprint.

Abstract: A large class of inverse problems for PDEs are only well-defined as mappings from operators to functions. Existing operator learning frameworks map functions to functions and need to be modified to learn inverse maps from data. We propose a novel architecture termed Neural Inverse Operators (NIOs) to solve these PDE inverse problems. Motivated by the underlying mathematical structure, NIO is based on a suitable composition of DeepONets and FNOs to approximate mappings from operators to functions. A variety of experiments are presented to demonstrate that NIOs significantly outperform baselines and solve PDE inverse problems robustly, accurately and are several orders of magnitude faster than existing direct and PDE-constrained optimization methods.

Abstract: We consider the ill-posed inverse problem of identifying a nonlinearity in a time-dependent PDE model. The nonlinearity is approximated by a neural network, and must be determined alongside unknown parameters and state. Proposing an all-at-once approach, we bypass the need for training data, and recover all unknowns simultaneously. Generally, approximation via neural networks can be realized as a discretization scheme, and training with noisy data can be seen as an ill-posed inverse problem. Therefore, we study discretization of regularization in terms of Tikhonov and projected Landweber methods, proving convergence as discretization error and noise level tend to zero.

This is joint work with Barbara Kaltenbacher.

Abstract: Characterizing the function spaces corresponding to neural networks can provide a way to understand their properties.  The talk is devoted to show how the theory of reproducing kernel Banach spaces can be used to characterize the function spaces corresponding to neural networks. In particular, I will show a representer theorem for a class of reproducing kernel Banach spaces, which includes one hidden layer neural networks of possibly infinite width. Furthermore, I will prove that, for a suitable class of ReLU activation functions, the norm in the corresponding reproducing kernel Banach space can be characterized in terms of the inverse Radon transform of a bounded real measure. 

The talk is based on joint work with F. Bartolucci, L. Rosasco and S. Vigogna.

Abstract: Source conditions are a key tool in variational regularisation to derive error estimates and convergence rates for ill-posed inverse problems. In this talk, we provide a recipe to practically compute source condition elements as the solution of convex minimisation problems that can be solved with first-order algorithms. We demonstrate the validity of our approach by recovering an image from a subset of the coefficients of its Fourier transform. We further demonstrate that the proposed approach can easily be modified to solve the machine learning task of identifying the optimal sampling pattern in the Fourier domain for given image and variational regularisation method, which has applications in the context of sparsity promoting reconstruction from magnetic resonance imaging data. This is joint work with Tatiana Bubba, Luca Ratti and Danilo Riccio.

Abstract: In work by Bora et al. (2017), a mathematical framework was developed for compressed sensing guarantees for random Gaussian sampling signals lying in the range of a generative neural network (GNN). Motivated by applications in medical imaging, in this talk we consider measurement matrices obtained by randomly subsampling a unitary matrix (e.g., subsampled DFT) and present the first known recovery error bounds for compressed sensing with GNNs and subsampled isometries. Our theory relies on the notion of coherence, a new parameter measuring the interplay between the range of the network and the measurement matrix. We also propose a regularization strategy for training GNNs to have favourable coherence with the measurement operator. In addition, we provide compelling numerical simulations that support this regularized training strategy: our approach yields low coherence networks that require fewer measurements for signal recovery. This, together with our theoretical results, supports coherence as a natural quantity for characterizing generative compressed sensing with subsampled isometries. 

This is a joint work with A. Berk (McGill) and B. Joshi, Y. Plan, M. Scott, X. Sheng, and O. Yilmaz (University of British Columbia)

Abstract: Recently, there has been significant interest in operator learning, i.e. learning mappings between infinite dimensional function spaces. This has been particularly relevant in the context of learning partial differential equations from data. However, it has been observed that proposed models may not behave as operators when implemented on a computer, questioning the very essence of what operator learning should be. We contend that in addition to defining the operator at the continuous level, some form of continuous-discrete equivalence is necessary for an architecture to genuinely learn the underlying operator, rather than just discretizations of it. To this end, we propose to employ frames, a concept in applied harmonic analysis and signal processing that gives rise to exact and stable discrete representations of continuous signals. Extending these concepts to operators, we introduce a unifying mathematical framework of Representation equivalent Neural Operator (ReNO) to ensure operations at the continuous  and discrete level are equivalent. Lack of this equivalence is quantified in terms of aliasing errors. We analyze various existing operator learning architectures to determine whether they fall within this framework, and highlight implications when they fail to do so.

Abstract: We will focus on data-driven regularization by projection showing some numerical results and comparisons with different methods. We will also discuss in detail “Seidman's non-convergence example on regularization”. Additionally, we provide an application of a projection algorithm, utilized and applied in frames theory, as a data driven reconstruction procedure in inverse problems.

Abstract: Due to its remarkable success for a variety of complex image processing problems, Deep Learning (DL) is nowadays also more commonly used in the field of X-ray Computed Tomography (CT). In this talk, we will highlight some of the challenges and potential solutions of integrating Deep Learning into tomographic work-flows found in scientific, clinical or industrial applications. In particular, we will cover the acquisition of large-scale experimental data collections suitable for DL and advanced topics such as online learning for real-time X-ray CT and adaptive projection angle selection using reinforcement learning. 

Abstract: The new wave of artificial intelligence is impacting industry, public life, and the sciences in an unprecedented manner. It has by now already led to paradigm changes in several areas. However, one current major drawback is the lack of reliability.

In this lecture we will first provide an introduction into this vibrant research area. We will then present some recent advances, in particular, concerning optimal combinations of traditional model-based methods with AI-based approaches in the sense of true hybrid algorithms, with a particular focus on limited-angle computed tomography and a novel approach coined "Deep Microlocal Reconstruction". Due to the importance of explainability for reliability, we will also touch upon this area by highlighting an approach which is itself reliable due to its mathematical foundation. Finally, we will discuss fundamental limitations of deep neural networks and related approaches in terms of computability, and how these can be circumvented in the future, which brings us in the world of quantum computing.

Abstract: We live in an era of  data-driven approaches to image analysis, where modeling is sometimes considered obsolete. I will propose in this talk giving back to accurate physical models of image formation their rightful place next to machine learning in the overall processing and interpretation pipeline, and discuss two applications:  super-resolution and high-dynamic range imaging from raw photographic bursts, and exoplanet detection and characterization in direct imaging at high contrast.

This is joint work with Theo Bodrito, Yann Dubois de Mont-Marin, Thomas Eboli, Olivier Flasseur, Anne-Marie Lagrange, Maud Langlois, Bruno Lecouat and Julien Mairal. 

Abstract: In this presentation I will present some recent developments in the field of optimisation and learning for super-resolution microscopy. To start with, I will present a model-aware generative adversarial framework for reconstructing high-resolution images from a temporal sequence of low-resolution stochastically fluctuating blurred data. Next, I will discuss how theoretically-grounded plug & play proximal denoisers can be effectively used to learn implicitly the unknown distribution of data, thus overcoming model-based reconstruction artefacts.

Abstract: TBA

Abstract: Motivated by classical work on the numerical integration of ordinary differential equations we present a ResNet-styled neural network architecture that encodes non-expansive (1-Lipschitz) operators, as long as the spectral norms of the weights are appropriately constrained. This is to be contrasted with the ordinary ResNet architecture which, even if the spectral norms of the weights are constrained, has a Lipschitz constant that, in the worst case, grows exponentially with the depth of the network. Further analysis of the proposed architecture shows that the spectral norms of the weights can be further constrained to ensure that the network is an averaged operator, making it a natural candidate for a learned denoiser in Plug-and-Play algorithms. Using a novel adaptive way of enforcing the spectral norm constraints, we show that, even with these constraints, it is possible to train a performant denoiser. The proposed architecture is applied to the problem of adversarially robust image classification and to imaging inverse problems.

Abstract: Infimal convolutions of two or more convex regularisers have proven very useful for decomposing an image into a sum of two or more components, each of which has low energy with respect to one of the functionals. Such components can, e.g., have different smoothness or different dominant directions of anisotropy. We extend this idea further to infimal convolutions of an infinite (continuum) family of functionals. For variational problems regularised by an infinite infimal convolution functional we propose a lifting to the space of vector-valued measures, where we prove a representer theorem (existence of sparse solutions in the case of finite-dimensional measurements) and derive a generalised conditional gradient method. This allows us to learn parameters of the infinite infimal convolution that best suit a given image. We apply this paradigm for learning dominant directions of oscillation in an image using an infinite infimal convolution of anisotropic fractional Laplacians.

Joint work with Kristian Bredies, Marcello Carioni, Martin Holler, and Carola Schönlieb.

Abstract: We consider the invariant manifold learning (that is, the geometric Whitney problem) on how a Riemannian manifold can be constructed to approximate a given discrete metric space. This problem is closely related to invariant manifold learning, where a Riemannian manifold  $(M,g)$ needs to be approximately constructed from the noisy distances $d(X_j,X_k)+\eta_{jk}$ of points  $X_1,X_2,\dots,X_N$, sampled from the manifold $M$. Here, $d(X_j,X_k)$ are the distance of the points $X_j,X_k\in M$ and $\eta_{jk}$ are either deterministic or random measurement errors. To study this problem we consider also learning of submanifolds of the high dimensional Euclidean spaces.

We also consider applications of the results in inverse problems encountered in medical and seismic imaging. In these problems, an unknown wave speed in a domain needs to be determined from indirect measurements. In geometric terms, this corresponds to the reconstruction of the Riemannian metric associated with the wave velocity from the wave kernel (or the heat kernel) measured in a subset of the domain. The presented results have been done in collaboration with Charles Fefferman, Sergei Ivanov, Yaroslav Kurylev, and Hariharan Narayanan.