PDE learning and neural networks

The PDEs are popular models for various phenomena. Unveiling the underlying laws in PDE forms from the data has been an active research topic in the machine learning community. However, the actual setting of the problem fits in the framework of typical inverse problems.  Many computational works based on regression emphasize "interpretability" in terms of sparsity (or certain penalization from domain knowledge). At the same time, there is very little attention to the feasibility (well-posedness) of the proposed problem, that is, does the data include sufficient information for the learning task?  Neural networks are widely used in various fields of research. Although the flexibility enables small approximation error for general approximation, it also complicates the training process. Usually, the oscillatory functions are complicated to approximate with the networks due to the implicit integral geometry of activation. 

Related work: 

1. How much can one learn a partial differential equation from its solution?  [with Y. He and H. Zhao], FoCM (2023), 1-47. [pdf]

2. How much can one learn from a single solution of a PDE? [with H. Zhao], accepted by Pure and Applied Functional Analysis, 2022. [pdf][code] 

3. Why shallow networks struggle with approximating and learning high frequency: a numerical study [with S. Zhang, H. Zhao, H. Zhou], Information and Inference: A Journal of the IMA 14 (3). [pdf]

4. Structured and balanced multi-component and multi-layer neural networks [with S. Zhang, H. Zhao, H. Zhou], to appear on SIAM Journal on Scientific Computing. [pdf][code]

5. Fourier multi-component and multi-layer neural networks: unlocking high-frequency potential [with S. Zhang, H. Zhao, H. Zhou], preprint. [pdf][code]

6. What Can One Expect When Solving PDEs Using Shallow Neural Networks? [with Y. He, H. Zhao, Y. Liang], preprint. [pdf]

Intrinsic complexity of models and data sets

Big data is a popular topic nowadays. Despite the high dimension of the phase space, it has been widely believed that the nature of data is approximately confined to a low-dimensional space. The intrinsic complexity is studying the minimal dimensions or parameters to describe the data with sufficient accuracy. In the linear space, the problem is equivalent to the Kolmogorov n-width, which is to find the best linear space to approximate the data.  For special settings, such as a random field or sampled random vectors, the intrinsic complexity will depend on the covariance function's smoothness and the scales. For integral equations derived from physical systems, such as the Lippmann-Schwinger equation or Peierls integral equation, the intrinsic complexity will depend on the separability of the integral kernels.  In the inverse problem, the inherent complexity has a strong relationship with the instability estimate. 

Related work:

1. Separability of the kernel function in an integral formulation for anisotropic radiative transfer equation [with H. Zhao and K. Ren], SIAM J.  Math Anal., 53 (5), 5613-5645, 2021. [pdf]

2. Intrinsic complexity and scaling laws: from random fields to random vectors [with J. Bryson and H. Zhao], Multiscale Model. Simul., 17(1), 460-481, 2019. [pdf]

3. Instability of the inverse problem for the stationary radiative transport equation [with H. Zhao],  SIAM J. Math Anal., 51(5), 3750-3768, 2019. [pdf]

4. Robustness of data-driven approaches in limited angle tomography [with  Y. Wang], SIAM Journal on Imaging Sciences 18 (1), 345-358 [pdf][code]

Imaging with physical models

The transport model has found its applications in various areas, including biomedical imaging, astronomy, nuclear engineering, remote sensing, seismology, and oceanology, among others. Depending on the medium's properties, the model could live on different scales. The most common type of inverse transport problem consists of reconstructing the medium's parameters from the boundary measurements (Albedo operator or its variants, travel times).  Under the diffusion regime, that is, the mean-free-path is much smaller than the medium's characteristic length, the transport model is often reduced to the diffusion model.  

Related work:

1. Inverse transport problems in quantitative PAT for molecular imaging [with K. Ren and R. Zhang], Inverse Problems, 31, 125012, 2015. [pdf]

2. A hybrid inverse problem in the fluorescence ultrasound modulated optical tomography in diffusive regime [with W. Li and Y. Yang], SIAM J. Appl. Math., 79(1), 356-376, 2019. [pdf]

3. A hybrid adaptive phase space method for reflection traveltime tomography [with H. Zhao], SIAM J. Imaging Sci., 12(1), 28-53, 2019. [pdf]

4. An inverse transport problem in the fluorescence ultrasound modulated optical tomography [with W. Li and Y. Yang],  Inverse Problems, 2019. [pdf]

5. Unique determination of absorption coefficients in a semilinear transport equation [with K. Ren], SIAM J. Math. Anal., 53(5). [pdf]

6. Quantitative PAT with simplified PN approximation [with H. Zhao], Inverse Problems, 37 (5), 055009.  [pdf]

7. Inverse boundary problem for the two-photon transport equation [with P. Stefanov], SIAM J. Math Anal 54.3 (2022): 2753-2767. [pdf]

8. Acousto-electric Inverse Source Problems [with W. Li,  J. Schotland and Y. Yang], SIAM J. Imaging Sci., 14 (4), 1601-1616. [pdf] 

9. Transport models for wave propagation in scattering media with nonlinear absorption [with J. Kraisler, W. Li,  K. Ren,  J. Schotland], SIAM J. Applied Math 83.4 (2023), 1677 - 1695. [pdf]

10. Inverse source problem in ultrasound modulated Maxwell equations  [with W. Li, J. Schotland, Y. Yang], SIAM J. Applied Math. 83.2 (2023), 418-435. [pdf]

11. Reconstruction of acoustic and optical properties in PAT/TAT with data from multiple illuminations [with K. Ren],  preprint.

12. Forward and inverse problems of a semilinear transport equation [with K. Ren], preprint. [pdf]

Computations of radiative transport equation

The computation of the radiative transport equation (or even the Boltzmann equation) is regarded as a difficult task since it involves high dimensionality and has quite different natures under different settings.  The average lemma has shown that the angular space's integration has better regularity, which could reduce the complexity of the computation. In the simplest cases where the scattering phase function is isotropic or highly separable, it is possible to formulate an integral system with weakly singular kernels.  If the Krylov subspace method is used to solve the linear system, then fast algorithms could be utilized to accelerate. If the scattering phase function involves too many modes, then the resulting kernels are highly oscillatory, and the intrinsic numerical complexity will grow accordingly. 

Related work:

1. A fast algorithm for radiative transport in isotropic media [with K. Ren and R. Zhang],  Journal of Computational Physics, 2019. [pdf]

2. Separability of the kernel function in an integral formulation for anisotropic radiative transfer equation [with H. Zhao and K. Ren], SIAM J. Math. Anal, 2021. [pdf]

3. Fast numerical algorithm for radiative transport equation in time domain [with H. Zhao], CSIAM Trans. Appl. Math., 1 (2020) [pdf]

Implicit boundary integration

Numerically, the implicit boundary integral method provides a flexible quadrature rule depending on the surface on which the integral lives. Instead of on the surface, the discretization will live in a tube around the surface and be approximated through grid points. The method could be used to solve many quadrature-related problems, like integral equations on complicated surfaces. One of the applications is the macromolecular electronic potential. The macromolecular electronic potential is an essential quantity in molecular biology and chemistry. The electronic potential could be used to determine the nature of the molecule's chemical bond. In biology, a DNA-binding protein mainly possesses a pocket of positive charges, which could help to bind the DNA through its negative phosphate charges.  For interactions of two protein molecules, we have to look for a pocket of negative charges on one protein and a bump of positive charges on the other (or vice versa). Calculating the electronic potential map on the molecule could help to indicate a possible interaction site.   

Related work:

1. An implicit boundary integral method for computing electric potential of macromolecules in the solvent [with K. Ren and R. Tsai], Journal of Computational Physics 359 (2018): 199-215. [pdf]

2. Corrected trapezoidal rule-IBIM for linearized Poisson-Boltzmann equation [with F. Izzo, O. Runborg, R. Tsai], preprint. [pdf]

3. Error analysis for implicit boundary integral method [with K. Ren, R. Tsai, O. Runborg], BIT Numerical Mathematics 65 (1), 8 [pdf]

Besides the primary research topics, I keep expanding a variety of interests in other areas, including optimization, graph theory, random networks, dynamic systems, randomized algorithms, high-dimensional probability,  mathematical problems in data science, etc. I have a GitHub profile hosting my research codes, and I also used to keep a research blog on GitHub (stopped in  Jan. 2020 due to the pandemic).

In the following, I list a few questions (most are possibly open,  and some pages are not maintained since the Google site is not friendly to LaTeX) that I have been working on, thinking about, or am interested in. The underlying physics looks straightforward, but the rigorous mathematical theories are always more difficult.  

If you find any of these topics resolved or any of them interesting, please feel free to discuss them with me at yimin.zhong@auburn.edu.