COVID-19 transmission models in the real world: models, data, and policy
Forrest Crawford @ Yale (www.crawfordlab.io)
Abstract:
Simple mathematical models of COVID-19 transmission gained prominence in the early days of the pandemic. These models provided researchers and policymakers with qualitative insight into the dynamics of transmission and quantitative predictions of disease incidence. More sophisticated models incorporated new information about the natural history of COVID-19 disease and the interaction of infected individuals with the healthcare system, to predict diagnosed cases, hospitalization, ventilator usage, and death. Models also provided intuition for discussions about outbreaks, vaccination, and the effects of non-pharmaceutical interventions like social distancing guidelines and stay-at-home orders. But as the pandemic progressed, complex real-world interventions took effect, people everywhere changed their behavior, and the usefulness of simple mathematical models of COVID-19 transmission diminished. This challenge forced researchers to think more broadly about empirical data sources that could help predictive models regain their utility for guiding public policy. In this presentation, I will describe my view of the successes and failures of population-level transmission models in the context of the COVID-19 pandemic. I will outline the evolution of a project to predict COVID-19 incidence in the state of Connecticut, from development of a transmission model to engagement with public health policymakers and initiation of a new data collection effort. I argue that a new data source – passive measurement of close interpersonal contact via mobile device location data – is a promising way to overcome many of the shortcomings of traditional transmission models. I conclude with a summary of the impact this work has had on the COVID-19 response in Connecticut and beyond.
Bio:
Forrest W. Crawford is an Associate Professor of Biostatistics, Statistics & Data Science, Operations, and Ecology & Evolutionary Biology at Yale University. He is affiliated with the Center for Interdisciplinary Research on AIDS, the Institute for Network Science, the Computational Biology and Bioinformatics Program, and the Public Health Modeling Concentration. His research focuses on mathematical and statistical problems related to discrete structures and stochastic processes in epidemiology, public health, biomedicine, and social science. He received the NIH Director's New Innovator Award in 2016.
Summary:
Focus:
Analyzing the spread of COVID-19 in Connecticut
Methodology and interactions with public health and public policy experts
Observation:
Lack of federal coordination in COVID response
States and localities engaged their own experts and crafted their own COVID response policies
March 2020:
Goal of response
How to allocate testing, hospital readiness and supply
How to manage lock-downs
How to reopen
Approach: modeling effort predict outcomes of interventions, evaluate outcomes of prior interventions
Many available data streams (near real-time)
Tests performed
Positive test/cases/diagnoses
Hospitalizations
Deaths
Syndromic surveillance
Outbreak investigations, contact tracing
Bias:
Hospitalization admission criteria didn’t change much
Deaths were also largely unbiased
Testing (amounts, who does it) was very biased since it was the cause of case count data and was affected by case count data
SEIR models:
Susceptible -> Exposed -> Infected -> Removed
Model is very constrained: can only describe epidemics go up, then go down
Very popular among epidemiologists because simple models that are based on mechanistic intuitions are easy to appreciate and understand
More complex models (e.g. ARIMA, Gaussian processes, ML) are very rarely used
Experts agree on the major dynamics of epidemics in well-mixed populations
Desire for interpretability, identifiability
Data is available for non-identifiable parameters
Statistical uncertainty of estimated parameters
Concerns about time-varying confounding
Initial projections were based on the governor’s lockdown plans and their guesses about how interpersonal contact will be affected by the lockdown
Challenge: behavioral contact data was not available
Available data showed that mobility had returned to Feb 2020 baseline by June 2020
But reality was that people were behaving differently
They used phone location data to estimate inter-personal data
Many companies published travel pattern data (e.g. time spent away from home, distance traveled)
Unclear about how these map to personal contact
They instead looked for data on how often two devices were within 6 feet of each other
Basic model came out: prevalent infections + contact rates => more COVID
Policymakers focused on controlling contact rates
E.g.
Malls didn’t control contact and ended up spreading COVID
Casinos did control contact and were both safer and faced less scrutiny from policymakers
Mobility metrics varied a lot
Metrics based on distance traveled and cities visited didn’t correlate with contact
Metrics based on type of location visited (e.g. workplace, retail) did correlate well
Contact data informed policy but to a large extent Connecticut didn’t take preemptive action based on them
They were able to use contact data to explain outbreak events in the data and indoor/outdoor and masking didn’t seem to affect explainability