Modeling @ DARPA
Abstract:
In this talk I will summarize a program portfolio at DARPA that seeks to develop technologies and platforms for machine-assisted planning, quantitative analysis, data science, and scientific/expert modeling. Programs covered include:
Causal Exploration of Complex Operational Environments (CausX, https://www.darpa.mil/program/causal-exploration), which is developing a general purpose conceptual modeling platform to aid planners in understanding and addressing underlying factors that drive complex situations like global competition and conflict or disaster response and risk reduction.
World Modelers (https://www.darpa.mil/program/world-modelers), which aims to develop technology that integrates qualitative analyses with results from quantitative models and data analyses to provide a comprehensive understanding of complicated, dynamic systems. The program has focused on issues around food, water, and physical security in Sub-Saharan Africa and the present and future challenges posed by global trends like climate change.
Data-Driven Discovery of Models (D3M, https://www.darpa.mil/program/data-driven-discovery-of-models), which seeks to automate many of the difficult parts of the data-science workflow, with AutoML tools covering dozens of problem and data types that dramatically increase the productivity of expert data scientists via automation and intuitive interfaces that enable subject matter experts to create best-in-class empirical models without the need for expert data scientists.
Automating Scientific Knowledge Extraction and Modeling (ASKEM, https://www.darpa.mil/program/automating-scientific-knowledge-extraction), which seeks to develop tools for the agile creation and sustainment of complex models and simulators that support decision making in rapidly evolving complex scientific domains ranging from viral epidemics to the impacts of space weather.
AI-assisted Climate Tipping-point Modeling (ACTM, https://www.darpa.mil/program/ai-assisted-climate-tipping-point-modeling), which is exploring the use of modern data and simulation based AI approaches to improve the scientific understanding (and predictability) of potential future tipping-points in climate and climate-forcing, which represent potential threats and “strategic surprise” at a planetary scale.
Bio:
Dr. Joshua Elliott joined DARPA in September 2017. His research interests include modeling and prediction of complex natural and socio-economic systems and how computational technologies can be leveraged to improve all aspects of science and modeling from data discovery to analysis.
Elliott joins DARPA from the University of Chicago Computation Institute, where he led research projects related to socio-technical change, optimal decision- and policy-making under uncertainty, and environmental variability and its impact on food security. Elliott has also held positions with the Argonne National Lab, the London School of Economics and Political Science, and Columbia University.
Summary:
Machine-assister Planning/Design (causal exploration)
Design process is very manual and slow
Diverse teams, contrarian views, time to reach consensus
Tooling for supporting this process has not evolved (lots of sticky notes)
They are developing tools to document experts’ descriptions of the world in a GUI
Graph models
Attributes on nodes/edges
Computational objects that live on for future analysis
Facilitates planning, does not automate it
Makes it easy to document data sources and their implications
Can be converted into summary reports
Usable to both explore data/dynamics and analyze the evolution of scenarios
Machine-assisted “analysis” (world modelers)
Founded from tools to identify implications of existing research
E.g. Lum.ai is an outcome: “Lum specializes in novel natural-language-processing and machine-reading technology that helps top-tier seed solutions and agriscience companies quickly access ground-breaking industry discoveries.”
World modelers: expand dynamic analysis/inference to more complex systems
E.g. food security, refugees, climate change
Goal: identify unintended consequences of interventions
Challenge: systems are complex, densely connected, many actors
Analysts use their experience and mental models to create analysis frameworks
DOJO:
Prepare, publish and update models
Document model structure, input, training
Models represented as containers
CAUSEMOS:
Analyze dynamics of models and their implication
Output model runs into reports for stakeholders
Machine-assisted data science (data-driven discovery of models)
Domain SME Interfaces for data science
Problem understanding
Data preparation
Model
Model evaluation
Collaboration on model design
Customizable open-source AutoML Ecosystem
Programmatic interface
AutoML engines
Meta-learning
>300ML primitives for ML algorithms
Machine-assisted scientific modeling
Tools for agile creation and sustainment of complex models
Most simulation software is black-box
Code is a bad medium for communicating model structure
Vision: DevOps-style approach to scientific modeling and knowledge sharing
Synergistic Discovery and Design
Protens Design
Perovskite Design
Yeast Circuit
…