Opportunities for Machine Learning in Human Health

Abstract:

Understanding real-world dynamical phenomena remains a challenging task. In human health, as much as in many other scientific disciplines, machine learning has advanced as the go-to technology to analyze nonlinear dynamical systems, identify patterns in big data, and make decisions around them. This seminar systematically compares two families of machine learning tools and illustrates their applications in human health: neural networks and Bayesian inference. Neural networks minimize a loss function to optimize the network parameters, without any prior knowledge of the underlying physics. Physics informed neural networks expand the loss function by an additional physics term and not only interpolate the training data, but also extrapolate and predict future behavior. Bayesian inference maximizes a prior-weighted likelihood function to estimate posterior distributions of model parameters. It not only infers model parameters to fit the training data, but also provides credible intervals to quantify the quality of the model. In this talk, we illustrate the potential of neural networks, Bayesian inference, and a combination of both for dynamical systems in human health. We discuss applications to the COVID-19 pandemic, the human heart, and the aging brain with the goal to generalize physics-informed machine learning to a wide variety of nonlinear dynamical systems and open new opportunities for machine learning in the benefit of human health.

Bio:
Ellen Kuhl is the Walter B. Reinhold Professor in the School of Engineering and Robert Bosch Chair of Mechanical Engineering at Stanford University. She received her PhD from the University of Stuttgart in 2000 and her Habilitation from the University of Kaiserslautern in 2004. Her area of expertise is Living Matter Physics, the design of theoretical and computational models to simulate and predict the behavior of living systems. Ellen has published more than 200 peer-reviewed journal articles and edited two books; she is an active reviewer for more than 50 journals at the interface of engineering and medicine and an editorial board member of seven international journals in her field. During the COVID-19 pandemic, she published a textbook on Computational Epidemiology and Data-Driven Modeling of COVID-19. Ellen is a founding member of the Living Heart Project, a translational research initiative to revolutionize cardiovascular science through realistic simulation with 400 participants from research, industry, and medicine from 24 countries. She is the current Chair of the US National Committee on Biomechanics and a Member-Elect of the World Council of Biomechanics. She is a Fellow of the American Society of Mechanical Engineers and of the American Institute for Mechanical and Biological Engineering. She received the National Science Foundation Career Award in 2010, the Humboldt Research Award in 2016, and the ASME Ted Belytschko Applied Mechanics Award in 2021. Ellen is a three-time All American triathlete, a multiple Boston, Chicago, and New York marathon runner, and a two-time Kona Ironman World Championship qualifier.

Summary:

Big data is the new big opportunity for scientists
Challenges of modeling from data
- Interpolation
- Extrapolation
- Identification/explanation
Technique: Neural networks
- Challenge: under/over-fitting
- The quality of interpolation increases with more model layers, but extrapolation is poor and the model explains nothing
- Physics-based modeling: example from COVID forecasting
  - e.g. harmonic oscillator has an ODE and a few parameters
    - Can be solved analytically
  - Can fit the oscillator ODE to data
  - Or can add the ODE to a neural network
    - Neural network predicts the outcome, AND the parameters of the ODE
    - ODE makes a prediction
    - Neural network’s loss function penalizes
      - Prediction error of the neural net
      - Prediction error of the ODE
      - Disagreement between neural network and ODE
  - Interpolation error is now larger but extrapolation error is smaller in the COVID forecasting dataset
  - Can look at ODE parameters to get an intuitive understanding of how the dynamics should work
- Physics-based modeling: Heart simulation
  - Challenge: small dataset
  - Physics: eikonal equation of wave propagation
  - Approach: PINNs (Physics Informed Neural Nets)
    - Sample at a few points in space
    - Learn a decent neural net from those
Technique: Bayesian inference
- Naturally provide confidence intervals around predictions
- COVID forecast:
  - Base model is SEIR, enhanced with
    - dynamic contact rate (lock-downs and political measures affect this)
    - dynamic reproduction number (virus changes)
  - Can fit this from data on personal mobility, virus reproduction
- Alzeheimer’s disease
  - Cause: amyloid-beta and tau proteins are mis-folded
  - Dynamics: first amyloid-beta increases, then tau, then brain shrinking, all following sigmoid shapes
  - Cognitive impairment follows changes in proteins by decades
  - Can we enable earlier detection by looking at mis-folder proteins?
  - Model:
    - Reaction-diffusion equation
    - Network diffusion equation
    - Different levels of precision: 3D Finite-elements, 2D finite elements or an interaction graph
      - Last is cheapest but good enough for this purpose
    - Train the parameters of this model from data using Markov-Chain Monte Carlo
    - They have a hierarchical prior:
      - Groups: different prob distributions
      - Subjects: different prob distributions
      - Graph diffusion brain model
    - Can
      - Separate the healthy and Alzeheimer’s groups based on protein levels
      - Differentiate rates of brain shrinking among healthy (1%/year) and Alzeheimer’s brains (1.5%/year)
      - Divide impacts by different tissues, which makes it possible to understand impact on the different cognitive abilities provided by these tissues
Bayesian neural networks
- Learn distributions of neural network parameters using Bayes
- Constrain network outputs using physics equations
New direction: Constitutive Neural Network
- E.g. put a strain into the network and pull out a stress
- Preprocess inputs to network to reflect known invariants
- Ensure that outputs to the network are quantities that mean invariant things
  - E.g. if we know something has rotational invariance then the data and model should be just 1D in the direction from the center and that’s baked into the problem formulation
- Assign specific physical meaning to the internal state of the neural network
  - Can add use invariants among these internal variables
    - E.g. symmetries, incompressibility, conservation
- Can enforce convexity of the results