----Special Session on the AMS 2024 Spring Central Sectional Meeting
April 20-21, 2024 (Saturday - Sunday), University of Wisconsin-Milwaukee, Milwaukee, WI
Organizers: Dexuan Xie, University of Wisconsin-Milwaukee; Zhen Chao, Western Washington University
Recent advancements in deep learning have promoted the development of deep learning methods to tackle previously insurmountable partial differential equation (PDE) problems, surpassing the capabilities of traditional finite difference, finite element, and spectral methods. This special session will invite distinguished professors and outstanding postdocs to deliver forty-five-minute talks. An additional ten minutes is given to each talk for questions and discussions. Through these talks, attendees will gain insights into the latest deep learning-based numerical PDE solvers and their applications across various scientific domains, such as fluid dynamics, ion channel simulations, optimal investment problems, image recognition problems, inverse problems, etc.
8am-9am, Lu Lu, Yale University: talk slides
Title: Accurate, Efficient, And Reliable Learning Of Deep Neural Operators For Multiphysics And Multiscale Problems
Abstract: It is widely known that neural networks (NNs) are universal approximators of functions. However, a less known but powerful result is that a NN can accurately approximate any nonlinear operator. This universal approximation theorem of operators is suggestive of the potential of deep neural networks (DNNs) in learning operators of complex systems. In this talk, I will present the deep operator network (DeepONet) to learn various operators that represent deterministic and stochastic differential equations. I will also present several extensions of DeepONet, such as DeepM&Mnet for multiphysics problems, DeepONet with proper orthogonal decomposition or Fourier decoder layers, MIONet for multiple-input operators, and multifidelity DeepONet. I will demonstrate the effectiveness of DeepONet and its extensions to diverse multiphysics and multiscale problems, such as bubble growth dynamics, high-speed boundary layers, electroconvection, hypersonics, geological carbon sequestration, and full waveform inversion. Deep learning models are usually limited to interpolation scenarios, and I will quantify the extrapolation complexity and develop a complete workflow to address the challenge of extrapolation for deep neural operators.
9am-10am, Yingjie Liu, Georgia Institute of Technology: talk recording
Title: Neural Networks With Local Converging Inputs (NNLCI) For Solving Conservation Laws With Orders Of Magnitude Reduction In Complexity And Training Cost
Abstract: This talk is based on a series of joint works (Communications in Computational Physics, 34 (2023), pp. 290-317 and pp. 907-933) with Haoxiang Huang and Vigor Yang. We are able to predict discontinuities and smooth parts of solutions of the Euler equations in 1D and 2D by a neural network accurately. For example, in order to predict the solution of the 1D Euler equations at a space-time location, one can design the output of a neural network to be the solution value at the location. If one tries to design the input as the low-cost numerical solution patch in a local domain of dependence of the location (where the information comes from), can the neural network tell if the input is across a shock or in a smooth region ? The answer is no ! Our approach uses two numerical solutions of a conservation law from a converging sequence, computed from low-cost numerical schemes, and in local domain of dependence of a space-time location as the input for a neural network to predict its high-fidelity solution at the location. Despite smeared input solutions, the output provides sharp approximations to solutions containing shocks and contact discontinuities. The method reduces the complexity (compared to a fine grid numerical simulation) and training cost by one or two orders of magnitude because it’s a local method, and is expected to have much more savings on complexity and training cost in higher dimensions or for smooth solutions.
10am -11am, Jonathan Siegel, Texas A&M University: talk recording
Title: Approximation Rates For ReLU^k Neural Networks On Sobolev And Besov Spaces
Abstract: In this work, we study how efficiently (in terms of the number of parameters) neural networks with the ReLU^k activation function can approximate functions from Sobolev spaces. This is a fundamental question for the application of neural networks in scientific computing. Specifically, we will consider approximating functions from the Sobolev space W^s(L_q) with error measured in the L_p-norm. Building upon existing work, we present optimal rates for very deep networks, which show that deep neural networks can outperform traditional methods on this task. However, this improvement comes at the cost of non-encodable parameters and may not be practical to implement. Further, we consider determining optimal rates for neural networks with a single hidden layer, for which we determine nearly optimal rates in the linear regime p<=q. A surprising consequence is that although shallow ReLU^k networks represent piecewise polynomial functions of fixed degree k, they can capture smoothness of significantly higher order s<=k+(d+1)/2 depending upon the dimension. This shows that the non-linear nature of the approximation produces a high-order method based upon low-order piecewise polynomials. Finally, we will discuss extensions to Besov spaces and remaining open problems.
3pm-4pm, Yue Yu, Lehigh University: talk recording
Title: Synergizing Imperfect Physics Knowledge And Measurements With Neural Operators
Abstract: Over the past decades, constitutive models within PDE-based continuum mechanics have been commonly employed to characterize material response from experimental measurements. However, in this approach accuracy and computational feasibility can be compromised when the physics knowledge is limited, e.g., when the governing law remains unknown. On the other hand, machine learning (ML) based approaches have emerged to provide more flexible predictive models. However, the pure data-driven approaches generally do not guarantee fundamental physical laws. As a result, their performances highly rely on the quantity and coverage of available data. In this talk, we develop physics-guided neural network models to mitigate the above challenges in pure physics-based and data-driven approaches. The key idea is to encode partial physical laws and PDE solving techniques through neural network architecture design. As such, the learnt model automatically preserves fundamental physical laws while still being readily applicable to learn physical systems directly from experimental measurements. To this end, we employ nonlocal neural operators, which learns a surrogate mapping between function spaces and acts as implicit solution operators of hidden governing PDE equations. We showcase the proposed physics-guided neural operators in preserving objectivity, momentum balance laws, and conservation laws. This feature enhances the model’s generalizability, especially in small and noisy data regimes.
4pm-5pm, Weimin Han, University of Iowa: talk recording
Title: Models, Analysis, And Numerical Solution Of Hemivariational Inequalities In Fluid Mechanics
Abstract: Recently, studies of hemivariational inequalities have attracted much attention in the research communities. Hemivariational inequalities are generalizations of variational inequalities. Through the formulation of hemivariational inequalities, application problems involving nonmonotone, nonsmooth and multivalued constitutive laws, forces, and boundary conditions can be treated successfully. In this talk, we introduce models of Stokes and Navier-Stokes hemivariational inequalities in fluid mechanics, and present recent and new results on analysis and numerical solution of the hemivariational inequalities.
5pm-6pm, Jiequn Han, Flatiron Institute: talk recording
Title: Enjoy The Best Of Both Worlds: A Neural-Network Warm-Start Approach For PDE Problems
Abstract: Partial differential equations (PDEs) are among the most ubiquitous tools in modeling natural phenomena, and various numerical methods have been developed for decades to solve PDE problems. While deep learning has introduced new techniques to the field, the limited accuracy of deep neural networks hinders their application to scientific problems that require high precision. This talk will present a warm-start approach that combines the strengths of deep neural networks and classical numerical solvers. The approach uses neural networks to provide an initial guess, enabling classical numerical solvers to achieve a good solution more efficiently. We will demonstrate the advantages of the proposed method through two examples. In the first example, we will demonstrate how the initial guess provided by neural networks helps identify the basin of attraction of the true solution in the inverse scattering problem. In the second example, we will show how the initial guess provided by neural networks leads to faster convergence in solving the Reynolds-averaged Navier-Stokes equations. In both examples, the combination of the classical PDE solver and the neural network outperforms either approach alone. The potential of this approach will be discussed, along with the new challenges that must be tackled to solve challenging scientific problems using this new paradigm.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
7pm-9pm, Dinner, Sze Chuan Restaurant at 11102 W National Ave, West Allis, WI 53227
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
8am-9am, Wenrui Hao, Pennsylvania State University: talk recording
Title: Neural Network Discretization For Solving Nonlinear PDEs
Abstract: In this talk, I will provide a comprehensive overview of recent developments in the application of Neural Networks for solving nonlinear Partial Differential Equations (PDEs). Specifically, I will introduce the Gauss-Newton approach for solving nonlinear PDEs. This method not only accelerates computation but also enhances accuracy when compared to conventional gradient-based training algorithms. Additionally, I will present HomPINNs, a novel framework that combines Physics-Informed Neural Networks (PINNs) with the well-established homotopy continuation method. Within this innovative framework, we demonstrate its efficacy in efficiently solving nonlinear elliptic differential equations with multiple solutions. Our research findings underscore the scalability and adaptability of the HomPINN approach. This methodology serves as an effective tool for addressing differential equations featuring multiple solutions and unknown parameters. The presented results highlight the potential of HomPINNs as a valuable tool in solving complex nonlinear PDEs.
9am-10am, Panos Stinis, Pacific Northwest National Lab: talk slides
Title: Multifidelity Scientific Machine Learning
Abstract: In many applications across science and engineering it is common to have access to disparate types of data or models with different levels of fidelity. In general, low-fidelity data are easier to obtain in greater quantities, but it may be too inaccurate or not dense enough to accurately train a machine learning model. High-fidelity data is costly to obtain, so there may not be sufficient data to use in training, however, it is more accurate. A small amount of high-fidelity data, such as from measurements or simulations, combined with low fidelity data, can improve predictions when used together. The important step in such constructions is the representation of the correlations between the low- and high-fidelity data. In this talk, we will present two frameworks for multifidelity machine learning. The first one puts particular emphasis on operator learning, building on the Deep Operator Network (DeepONet). The second one is inspired by the concept of model reduction. We will present the main constructions along with applications to closure for multiscale systems and continual learning. Moreover, we will discuss how multifidelity approaches fit in a broader framework which includes ideas from deep learning, stochastic processes, numerical methods, computability theory and renormalization of complex systems.
10am-11am, Zezheng Song, University of Maryland: talk recording
Title: Finite Expression Method For Solving High-Dimensional PDEs
Abstract: Machine learning has revolutionized computational science and engineering with impressive breakthroughs, e.g., making the efficient solution of high-dimensional computational tasks feasible and advancing domain knowledge via scientific data mining. This leads to an emerging field called scientific machine learning. In this talk, we introduce a new method for a symbolic approach to solving scientific machine learning problems. This method seeks interpretable learning outcomes via combinatorial optimization in the space of functions with finitely many analytic expressions and, hence, this methodology is named the finite expression method (FEX). It is proved in approximation theory that FEX can avoid the curse of dimensionality in discovering high-dimensional complex systems. As a proof of concept, a deep reinforcement learning method is proposed to implement FEX for solving high-dimensional PDEs.
2pm-3pm, Ying Liang, Purdue University: talk recording
Title: Data-Assisted Algorithms For Inverse Random Source Problems
Abstract: Inverse source scattering problems are essential in various fields, including antenna synthesis, medical imaging, and earthquake monitoring. In many applications, it is necessary to consider uncertainties in the model, and such problems are known as stochastic inverse problems. Traditional methods require a large number of realizations and information on medium coefficients to achieve accurate reconstruction for inverse random source problems. To address this issue, we propose a data-assisted approach that uses boundary measurement data to reconstruct the statistical properties of the random source with fewer realizations. We compare the performance of different data-driven algorithms under this framework to enhance the initial approximation obtained from integral equations. Our numerical experiments demonstrate that the data-assisted approach achieves better reconstruction with only 1/10 of the realizations required by traditional methods. Among the various Image-to-Image translation algorithms that we tested, the pix2pix method outperforms others in reconstructing well-separated inclusions with accurate positions. Our proposed approach results in stable reconstruction with respect to the observation data noise.
3pm-4pm, Peter Lu, University of Chicago: talk recording
Title: Simulating Quantum Dynamics Using Deep Generative Models
Abstract: Simulating the dynamics of quantum systems is critical to many applications in fundamental physics, material science, chemistry, and quantum computing. However, the high-dimensional nature of many-body quantum states implies exponential scaling in the worst case due to the curse of dimensionality. Fortunately, for many physically relevant systems, the underlying quantum state has low-dimensional structure which can be exploited to make simulations of quantum phenomena tractable. Generative models in modern machine learning implicitly represent high-dimensional probability distributions and provide fast sampling. Because the quantum wavefunction (or density matrix for open quantum systems) has properties similar to a high-dimensional probability distribution, generative models can be adapted to efficiently represent complex quantum states with hidden low-dimensional structure. In this talk, we will discuss recent work on simulating quantum dynamics by using deep generative models to implicitly represent the quantum state. These approaches may also generalize to other high-dimensional PDEs where the state can be interpreted as a probability distribution.