Titles and abstracts

Abstract: Scientific machine learning (SciML) has been taking the academic world by storm as an interesting blend of traditional scientific modeling with machine learning (ML) methodologies like deep learning. While traditional machine learning methodologies have difficulties with scientific issues like interpretability, and enforcing physical constraints, the blend of ML with numerical analysis and differential equations has evolved into a novel field of research which overcome these problems while adding the data-driven automatic learning features of modern machine learning. Many successes have already been demonstrated, with tools like physics-informed neural networks, universal differential equations, deep backward stochastic differential equation solvers for high dimensional partial differential equations, and neural surrogates showcasing how deep learning can greatly improve scientific modeling practice. Consequently, SciML holds promise for versatile application across a wide spectrum of scientific disciplines, ranging from the investigation of subatomic particles to the comprehension of macroscopic systems like economies and climates.

However, despite notable strides in enhancing the speed and accuracy of these methodologies, their utility in practical and specifically industrial settings remain constrained. Many domains within the scientific community still lack comprehensive validation and robustness testing of SciML approaches. This limitation is particularly pronounced when confronted with complex, real-world datasets emanating from interactions between machinery and environmental sensors as usually addressed in industry. Still if appropriately addressed, SciML with its promise to accelerate innovations and scientific discoveries by orders of magnitudes, offers unique opportunities to address the insatiable desire for faster and more accurate predictions in many fields.

This presentation is dedicated to exploring recent advancements in the implementation of SciML techniques. We will discuss how methodologies can be refined to ensure their practical viability and scalability, particularly in industrial sectors where digital and physical components converge.

Abstract: Electrochemical energy research is increasingly data-driven. Advancements in high throughput experiments (HTE) and user-friendly machine learning (ML) toolboxes are enabling the exploration of extensive electrochemical material spaces and predictive modelling of functional material properties. However, modern ML tends to prioritize complex models for slight improvements in predictive accuracy, often overlooking the generalizability and scientific foundation of these models. Such opaque models may perform well on training data but typically fail to elucidate the underlying principles of electrochemical operation.

Explainable machine learning (X-ML) employs models and techniques that enhance the transparency of ML predictions, aiding in model robustness, decision justification, and scientific validation. Symbolic regression (SR), a popular method within X-ML, generates interpretable equations to fit statistical relationships in data. The learned equations can be readily inspected for i) non-linearities, ii) interactions between variables, iii) minima/maxima and iv) behavior at relevant limits. In this talk, we will present our work on applying SR to model ion conductivity in a li-ion battery electrolyte. [1] The resulting model not only accurately predicts ion conductivity based on salt concentration, temperature, and solvent composition, but also reflects functional aspects from established thermodynamic limiting laws. We consider symbolic regression a promising approach for deriving interpretable models to learn physico-chemical principles from experimental data.

Abstract: We present novel neural network architectures designed to discover dynamical systems from physical data as well as solve dynamical systems rising from optimal control problems. At the core of the investigation are three interconnected neural network models: Symplectic Networks (SympNets), Poisson Neural Networks (PNNs), and GENERIC Formalism Informed Neural Networks (GFINNs). Collectively, these models represent an advancement in the integration of machine learning with physical modeling, potentially offering new ways to solve problems in system identification and optimal control.

Joint work with Zhen Zhang and Jerome Darbon, Division of Applied Mathematics, Brown University

Abstract: When learning neural network vector field approximations from trajectories of a dynamical system, temporal discretization’s are often used in the loss function. Implicit integrators require solving non-linear systems; however, we show that mono-implicit methods could be applied explicitly in the loss function. Within the class of mono-implicit Runge-Kutta we prove an order barrier for symplectic methods and a noise cancelling property of symmetric methods. Finally, we construct a scheme where the mean of multiple trajectories are combined to reduce sensitivity when learning dynamics from noisy data. We demonstrate the robustness of this approach on multiple chaotic Hamiltonian systems.

Abstract: Despite the recent flurry of work employing machine learning to develop surrogate models to accelerate scientific computation, the "black-box" underpinnings of current techniques fail to provide the verification and validation guarantees provided by modern finite element methods. In this talk we present a data-driven finite element exterior calculus for developing reduced-order models of multiphysics systems when the governing equations are either unknown or require closure. The framework employs deep learning architectures typically used for logistic classification to construct a trainable partition of unity which provides a notion of a control volume, with associated boundary operators and consequent div/grad/curl operators. This alternative to a traditional finite element mesh is fully differentiable and allows construction of a discrete de Rham complex with a corresponding Hodge theory. We demonstrate how models may be obtained with the same robustness guarantees as traditional mixed finite element discretization, with deep connections to contemporary techniques in graph neural networks. For applications developing digital twins where surrogates are intended to support real time data assimilation and optimal control, we further develop the framework to support Bayesian optimization of unknown physics on the underlying adjacency matrices of the chain complex. By framing the learning of fluxes via an optimal recovery problem with a computationally tractable posterior distribution, we are able to develop models with intrinsic representations of epistemic uncertainty. 

Abstract: In recent years, the applications of deep learning to scientific computing have attracted much attention under the name of scientific machine learning. Such methods have some unique features that classical computational methods do not have, and are expected to improve scientific computing significantly. Theoretically, due to the universal approximation properties of neural networks, sufficiently large neural networks are expected to be able to accurately approximate the solutions of partial differential equations. On the other hand, detailed error analysis, such as the numerical errors when using a neural network of finite size, is fragmentary, although various analyses have been carried out by many researchers. In this talk, we focus in particular on physics-informed neural networks (PINNs) for solving partial differential equations, and review and integrate a part of existing error analyses for PINNs. We also discuss the effectiveness of the method based on these results.

Abstract: In this talk we discuss  structure preservation and deep learning with applications to shape analysis. This is a framework for  treating complex data and obtain metrics on spaces of data. A computationally demanding task for estimating distances between shapes, e.g. in object recognition, is the computation of optimal reparametrizations. This is an optimisation problem on the infinite dimensional group of orientation preserving diffeomorphisms. We present various approaches to discretize this problem.

Abstract: Physical modelling is at the heart of science and engineering, with most models expressed naturally in partial differential equation (PDE) form. Solving most PDEs of importance, however, is analytically intractable and necessitates falling back on numerical approximation schemes. Neural PDE solvers, which replace hand-designed update rules with neural computations, have been shown to dramatically improve efficiency and are very easy to build, with applications ranging from dynamical systems modelling to turbulence simulation to weather prediction. All these simulators learn to mimic the dynamics of complex systems directly from data, obviating the need for complex hand-designed simulation rules, in-depth domain knowledge, or discrepancies between models and reality. However, the solution accuracy of current neural solvers is still too low to be competitive with classical methods, which can trade resolution for accuracy. Furthermore, for time-dependent PDEs there are no guarantees that long rollouts lead to stable dynamics. Why is this? What is being done? And where does this leave us? In this talk, I will outline what I believe to be two of the largest hurdles in the field of neural PDE solvers, what has been done and the challenges we still have ahead of us.

Abstract: Sampling weights and biases of neural networks from certain probability distributions at random allows us to solve many physics-related machine learning tasks with surprisingly good accuracy. In many cases, sampling and the subsequent solution of a linear system for the outer weights outperforms iterative, gradient-based optimization regarding training speed and accuracy by several orders of magnitude. We will discuss function approximation, solution of partial differential equations, approximation of neural operators, as well as benefits and drawbacks of the approach compared to classical numerical schemes.

Abstract: We prove a priori and a posteriori error estimates for physics-informed neural networks (PINNs) for linear PDEs. For the analysis, we propose an abstract framework in the common language of bilinear forms, and we show that coercivity and continuity lead to error estimates.  We analyze elliptic equations in primal and mixed form, elasticity, parabolic, hyperbolic and Stokes equations; and a PDE constrained optimization problem. The obtained estimates are sharp and reveal that the L2 penalty approach for initial and boundary conditions in the PINN formulation weakens the norm of the error decay. Finally, utilizing recent advances in PINN optimization, we present numerical examples that illustrate the ability of the method to achieve accurate solutions.

Joint work with M. Zeinhofer and K. A. Mardal.

Abstract: In this talk we will utilise neural networks as a tool to resolve sub-grid scale dynamics for the shallow water equation with a view to couple deterministic models between scales. In particular, we will focus on convolutional neural networks and discuss connections between our approach and learning dynamical systems with neural networks.

Abstract: We present and analyse a framework for constructing neural networks that are intrinsically symplectic. This framework unifies many symplectic neural networks (SNNs) from the literature and can be used to construct new ones. We also discuss application of SNNs to: system identification, forced systems (e.g., dissipation), time-symmetric Hamiltonians and PDEs.

Abstract: In a recent work by Sherry et al. we considered neural network architectures constructed by applying certain numerical integrators to neural ODEs. Based on the classical work by Dahlquist and Jeltsch on circle contractivity, we designed neural networks such that the layer map is a nonexpansive and even an averaged operator. We describe this approach and show applications to denoising where improved robustness to adversarial data perturbations can be observed. If time permits, we shall briefly discuss the generalisation to the case where input data and hidden layers are modeled as a Riemannian manifold.

Abstract: In this contribution  we will present recent results of studies of quantum mechanical many-body problems using deep learning methods. We will present a general class of machine learning algorithms called parametric matrix models.  

Deep learning and so-called neural quantum mechanical states have recently been applied to studies of quantum mechanical many-body problems with quite some success. In these studies, a neural network replaces a traditional variational ansatz for the trial wave function in Monte Carlo simulations. 

Parametric matrix models on the other hand are based on matrix equations, and the design is motivated by the efficiency of reduced basis methods for approximating solutions of parametric equations. The dependent variables can be defined implicitly or explicitly, and the equations may use algebraic, differential, or integral relations. Parametric matrix models can be trained with empirical data only, and no high-fidelity model calculations are needed. Here we will show some selected examples of applications of parametric matrix models to a series of different challenges that show their performance for a wide range of problems. For all the cases we will  discuss, parametric matrix models produce accurate results within a computational framework that allows for parameter extrapolation and interpretability.

Abstract: Optimized design of experiments (DOE) for parameter identification plays a pivotal role in advancing battery testing methodologies, which are critical for the development of efficient and reliable energy storage systems. Physics-informed Deep Operator Networks offer a promising approach to obtain surrogate models for electrochemical models that can be seamlessly integrated with classical DOE methodologies offering numerical speedup, methodological flexibility, and edge-device compatibility.

Abstract: Structure-preserving machine learning has recently become an active area of research. Popular models in this field, such as Hamiltonian neural networks, would typically require data on the system's momentum and this can be a limitation of the approach. Instead, we consider a method for learning the differential equations describing the dynamics of a forced Lagrangian system. The method requires time-series measurements of the system's position only and can learn external forces, e.g., dissipative frictional forces.

Abstract: A mesh motion model based on deep operator networks is presented. The model is trained on and evaluated against a biharmonic mesh motion model on a fluid—structure interaction benchmark problem and further evaluated in a setting where biharmonic mesh motion fails. The performance of the proposed mesh motion model is comparable to the biharmonic mesh motion on the test problems, while the computational cost is comparable to the much faster and simpler harmonic mesh motion model. 

Abstract: Traditional deep learning models can only be trained on singleresolution data. We propose a novel architecture, named DON-LSTM, which extends the deep operator network (DeepONet) with a long short-term memory network (LSTM). This architecture allows us to explicitly incorporate multi-resolution data in training, as well as provide accurate predictions over long-time horizons. DON-LSTM is significantly more accurate and requires fewer high-resolution samples than its vanilla counterparts.

Abstract: Rayleigh-Bénard convection, is a classic fluid dynamics problem, with applications in geophysical, astrophysical, and industrial flows. Fourier Neural Operator (FNO) leverages neural networks and Fourier analysis to efficiently model spatiotemporal dynamics in fluid systems, offering a promising avenue for accurate and scalable simulations. In this poster, first results on the application of FNO for tackling the Rayleigh-Benard convection equations is presented.

Abstract: The interaction of light with nanostructures exhibits complex optical effects including scattering, polarization change, absorption, diffraction, and others. Accurate prediction of these effects requires rigorous electromagnetic simulations to solve a scattering problem through numerical approximation of Maxwell’s equations. Traditional electromagnetic field (EMF) solvers are inefficient for large field-of-view simulations, while data-driven deep learning networks rely on a huge amount of expensive rigorously simulated or measured data. In our work, we use the alternative data-free solution in the form of an established physics-informed neural network (PINN) with U-Net architecture to accurately model the light diffraction from the nano-optical devices, specifically EUV lithographic masks and meta-surfaces. Employing a vectorial formulation of the 3D Maxwell's equations as a loss function, we train a PINN accurately to model near field, far field, and corresponding optical and imaging effects. In contrast to data-driven deep learning, PINN provides a data-efficient solution by learning only from the governing physics. The resulting PINN model can accurately simulate light scattering response for arbitrary geometries and illumination parameters within milliseconds without re-training providing a significant speed-up compared to a numerical solver based on the waveguide method. Ultimately, the developed universal PINN-based EMF solver opens up capabilities for fast lithographic imaging, nano-optical design, and solving optical optimization problems.

Abstract: In this contribution we present our recent results on a new class of neural networks for hyperbolic problems. We have proposed and rigorously analysed estimatable varation neural networks (EVNNs) that allow a computationally cheap estimate on the BV norm motivated by an auxiliary function space BMV.

We prove a universal approximation theorem for this class of networks and discuss possible implementations of EVNNs. We construct sequences of loss functionals for ODEs and scalar hyperbolic conservation laws for which a vanishing loss leads to convergence. Moreover we show existence of sequences of loss minimizing neural networks if the solution is an element of BMV. 

Several numerical test cases illustrate that it is possible to use standard techniques to minimize these loss functionals for EVNNs.