Abstract:
Understanding real-world dynamical phenomena remains a challenging task. In human health, as much as in many other scientific disciplines, machine learning has advanced as the go-to technology to analyze nonlinear dynamical systems, identify patterns in big data, and make decisions around them. This seminar systematically compares two families of machine learning tools and illustrates their applications in human health: neural networks and Bayesian inference. Neural networks minimize a loss function to optimize the network parameters, without any prior knowledge of the underlying physics. Physics informed neural networks expand the loss function by an additional physics term and not only interpolate the training data, but also extrapolate and predict future behavior. Bayesian inference maximizes a prior-weighted likelihood function to estimate posterior distributions of model parameters. It not only infers model parameters to fit the training data, but also provides credible intervals to quantify the quality of the model. In this talk, we illustrate the potential of neural networks, Bayesian inference, and a combination of both for dynamical systems in human health. We discuss applications to the COVID-19 pandemic, the human heart, and the aging brain with the goal to generalize physics-informed machine learning to a wide variety of nonlinear dynamical systems and open new opportunities for machine learning in the benefit of human health.
Bio:
Ellen Kuhl is the Walter B. Reinhold Professor in the School of Engineering and Robert Bosch Chair of Mechanical Engineering at Stanford University. She received her PhD from the University of Stuttgart in 2000 and her Habilitation from the University of Kaiserslautern in 2004. Her area of expertise is Living Matter Physics, the design of theoretical and computational models to simulate and predict the behavior of living systems. Ellen has published more than 200 peer-reviewed journal articles and edited two books; she is an active reviewer for more than 50 journals at the interface of engineering and medicine and an editorial board member of seven international journals in her field. During the COVID-19 pandemic, she published a textbook on Computational Epidemiology and Data-Driven Modeling of COVID-19. Ellen is a founding member of the Living Heart Project, a translational research initiative to revolutionize cardiovascular science through realistic simulation with 400 participants from research, industry, and medicine from 24 countries. She is the current Chair of the US National Committee on Biomechanics and a Member-Elect of the World Council of Biomechanics. She is a Fellow of the American Society of Mechanical Engineers and of the American Institute for Mechanical and Biological Engineering. She received the National Science Foundation Career Award in 2010, the Humboldt Research Award in 2016, and the ASME Ted Belytschko Applied Mechanics Award in 2021. Ellen is a three-time All American triathlete, a multiple Boston, Chicago, and New York marathon runner, and a two-time Kona Ironman World Championship qualifier.
Summary:
Big data is the new big opportunity for scientists
Challenges of modeling from data
Interpolation
Extrapolation
Identification/explanation
Technique: Neural networks
Challenge: under/over-fitting
The quality of interpolation increases with more model layers, but extrapolation is poor and the model explains nothing
Physics-based modeling: example from COVID forecasting
e.g. harmonic oscillator has an ODE and a few parameters
Can be solved analytically
Can fit the oscillator ODE to data
Or can add the ODE to a neural network
Neural network predicts the outcome, AND the parameters of the ODE
ODE makes a prediction
Neural network’s loss function penalizes
Prediction error of the neural net
Prediction error of the ODE
Disagreement between neural network and ODE
Interpolation error is now larger but extrapolation error is smaller in the COVID forecasting dataset
Can look at ODE parameters to get an intuitive understanding of how the dynamics should work
Physics-based modeling: Heart simulation
Challenge: small dataset
Physics: eikonal equation of wave propagation
Approach: PINNs (Physics Informed Neural Nets)
Sample at a few points in space
Learn a decent neural net from those
Technique: Bayesian inference
Naturally provide confidence intervals around predictions
COVID forecast:
Base model is SEIR, enhanced with
dynamic contact rate (lock-downs and political measures affect this)
dynamic reproduction number (virus changes)
Can fit this from data on personal mobility, virus reproduction
Alzeheimer’s disease
Cause: amyloid-beta and tau proteins are mis-folded
Dynamics: first amyloid-beta increases, then tau, then brain shrinking, all following sigmoid shapes
Cognitive impairment follows changes in proteins by decades
Can we enable earlier detection by looking at mis-folder proteins?
Model:
Reaction-diffusion equation
Network diffusion equation
Different levels of precision: 3D Finite-elements, 2D finite elements or an interaction graph
Last is cheapest but good enough for this purpose
Train the parameters of this model from data using Markov-Chain Monte Carlo
They have a hierarchical prior:
Groups: different prob distributions
Subjects: different prob distributions
Graph diffusion brain model
Can
Separate the healthy and Alzeheimer’s groups based on protein levels
Differentiate rates of brain shrinking among healthy (1%/year) and Alzeheimer’s brains (1.5%/year)
Divide impacts by different tissues, which makes it possible to understand impact on the different cognitive abilities provided by these tissues
Bayesian neural networks
Learn distributions of neural network parameters using Bayes
Constrain network outputs using physics equations
New direction: Constitutive Neural Network
E.g. put a strain into the network and pull out a stress
Preprocess inputs to network to reflect known invariants
Ensure that outputs to the network are quantities that mean invariant things
E.g. if we know something has rotational invariance then the data and model should be just 1D in the direction from the center and that’s baked into the problem formulation
Assign specific physical meaning to the internal state of the neural network
Can add use invariants among these internal variables
E.g. symmetries, incompressibility, conservation
Can enforce convexity of the results