Modeling in Subsurface Geophysics: Mathematics, Computation, and Machine Learning

Abstract:
Understanding the structure of Earth's subsurface is critical in many science, engineering, and business domains, from earthquake monitoring and forecasting to hydrocarbon exploration and carbon sequestration. Observations of seismic waves, from both anthropogenic and non-anthropogenic sources, yield many techniques for recovering models of subsurface structure. In this talk, I will introduce the concepts and challenges associated with reconstructing models of the subsurface through seismic tomography, imaging, and inversion. I will present the subsurface inversion workflow, from data acquisition, through modeling and computation, to interpretation and decision making. In this context, I will present the mathematics, computation, and intuition behind full-waveform inversion, an adjoint-based method for solving this inverse problem where the solution is constrained by the governing physics of the wave equation. Finally, looking forward, I will connect the mathematics and computation of FWI to related concepts in deep learning.

Bio:
Russell J. Hewett Russell J. Hewett is Assistant Professor of Mathematics and affiliate faculty in Computational Modeling and Data Analytics at Virginia Tech. His research interests are at the intersection of high-performance computing, inverse problems, and deep learning. Prior to returning to academia, he was a research scientist and R&D project manager for inverse problems, uncertainty quantification, and machine learning at TotalEnergies' Houston, TX research office, where he was architect for industrial-scale seismic inversion software and managed internal and external research projects. Russell was Postdoctoral Associate in mathematics and at the Earth Resources Laboratory at MIT, where he developed PySIT, a research and teaching tool for seismic inversion in Python. He is active in the scientific Python community and a member of the board of directors for the SunPy solar data analysis package as well as developer of DistDL, a distributed deep learning package in Python. He was a NASA Graduate Student Research Program fellow, earned his Ph.D. in computer science from the University of Illinois at Urbana-Champaign, with a focus on computational science and engineering, and was a visiting scholar at Trinity College in Dublin, Ireland. Recently, he received the Early Career Research Program award from the Office of Science in the Department of Energy for his research on extreme-scale parallelism in deep learning.

Summary:

Focus:
- Forward problem:
  - Solvers for differential equations
  - Parallel solvers
- Inverse problem:
  - Optimization of solutions to geological constraints / inverse problems
  - Using ML techniques for optimization
- Broadly:
  - Inverse solvers for other physics systems without clean physics formulations
    - Economics
    - Ecology
  - Uncertainty Quantification
Seismic Modeling Scales
- Global:
  - >104km scale
  - Modeling Earth core or plates
  - Sun subsurface, inference of state of opposite side of the sun
  - Mars (limited since only 1 seismic instrument)
- Continental
Key data: propagation of waves through sub-surface
- Look for
  - Travel time
  - Changes in wave frequency and amplitude
  - Wave class
- Sources:
  - Natural (e.g. earthquakes)
  - Anthropogenic (vibration or impact trucks)
    - Established networks of source arrays
    - Fixed of mobile arrays (e.g. marine surveys that)
      - Fixed: tend to look for earthquakes in different locations, so a fixed sensor will get a different angle to each earthquake
      - Mobile: change angle of impact and relative angle of impact and observation
Inverse problem
- Seismic Tomography:
  - Time wave’s travel time from source to destination
  - Good for recovering smooth sub-surface structures
  - Good for low frequency waves
- Seismic (Full-waveform) Inversion
  - Optimization of a seismic wave propagation model
  - Initial solution: output of tomography model, prior inversions
  - Good for many frequencies, especially middle range to which subsurface is not very sensitive
- Seismic Imaging:
  - Goal: recovery of derivative products of subsurface map
    - Reflectivity, edges of layers
  - Good for high-frequency waves
- Different approaches good for recovering different aspects subsurface
Adjoint
- Given differential equation can derive its adjoint, which is the equation’s gradient
- Key for optimizing/inverting model using a gradient descent
- Very related to automatic differentiation techniques
- Optimization produces the actual wave propagation, can be run backwards to create picture of subsurface
Challenges:
- Cycle skipping: some solutions can be moved by half a cycle and still produce the same results
- Solution:
  - Work in frequency space, partition simulations by frequencies
  - Very expensive
- Acoustic and elastic waves propagate at very different speeds
- Different subsurface materials also affect frequencies and propagation speeds
- Requires a lot of compute to resolve them all
Costs
- $100m for data
- 100s TB data
- 100s PF of compute
Opportunities for ML
- Full Waveform Inversion and ML have been used independently or coupled
- Are now being used fully integrated
- Can implement Full Waveform Inversion as a neural network, can get an easy derivative
- Training a neural network is an ODE-contrained optimization
- Neural network can be used as a prior for PDE solution or physical model
- Russell Hewett has been working on ways to do this efficiently and fully exploit parallelism