Science-Guided AI

Abstract: Computational seismic imaging is crucial for energy exploration, civil infrastructure, groundwater contamination and remediation, and so on. However, the relevant data analysis capability for solving computational seismic imaging problems is inadequate, mainly due to the ill-posed nature of the problems and the high computational costs. Recently, machine learning (ML) based computational methods have been pursued in the context of scientific computational imaging problems. Some success has been attained when an abundance of simulations and labels are available. Nevertheless, ML models, trained using physical simulations, usually suffer from weak generalizability when applied in a moderately different real-world dataset. Moreover, obtaining corresponding training labels is typically prohibitively expensive due to the high demand for subject-matter expertise. On the other hand, different from imaging problems from a typical computer vision context, many scientific imaging problems are governed by underlying physical equations. For example, the wave equation is the governing physics for seismic imaging problems. To fully unleash the power and flexibility of ML for solving large-scale computational seismic imaging problems, we have developed new computational methods to bridge the technical gap by addressing the critical issues of generalizability and data scarcity. In this talk, I will go through our recent R&D efforts in leveraging both the power of machine learning and underlying physics. A series of numerical experiments are conducted using datasets from synthetic simulations to field applications to evaluate the effectiveness of our techniques.

Bio: Youzuo Lin is a Computational Scientist at the Earth and Environmental Sciences Division of Los Alamos National Laboratory. Before joining as a staff scientist at LANL, he completed his Ph.D. in Applied and Computational Mathematics at Arizona State University. His current research focuses on physics-informed machine learning, deep learning, computational methods, and their applications in computational imaging, signal and image analysis. Specifically, he has worked on subsurface imaging for energy exploration, medical imaging and cancer detection, and time series classification for small earthquake detection.

Abstract: We are heading towards a cyber-physical age, where increasingly the engineered physical systems of the past such as transportation infrastructure, power infrastructure, buildings and factories are being integrated with computation and communication systems for the goals of autonomous operation, optimization, maintenance and personalization. These infrastructures are critical to modern life and any disruption or inefficiency in their operation has economic, environmental and societal costs. Given their complexity, AI and machine learning are important tools for achieving the above goals. However, before AI can be deployed widely in real-world critical cyber-physical systems (CPS), it needs to be made more robust. Some of the key challenges include: 1) How can an AI model (e.g., a CPS anomaly detection model) explain its output? 2) How can data be used for model induction while preserving data privacy? 3) How can domain knowledge be incorporated into data-driven models? In this talk, I'll present some ongoing and past work on applications of AI to cyber-physical systems.

Bio: Manish Marwah is a principal research scientist in a research group at Micro Focus (until recently part of Hewlett Packard Labs). His main research interests are in the broad areas of AI and data science, especially in the context of cyber-physical systems, and applications to cyber-security. He has initiated and led research on designing data science methods for sustainability and energy management of smart buildings and data centers. Recently, he has been working on large-scale analytics and its applications to IoT and security domains. His research has led to over 65 refereed papers, several of which have won awards, including at AAAI, KDD, and IGCC. He has twice co-organized -- Data Mining for Sustainability (SustKDD), a workshop at KDD. He has been granted 43 patents (with several pending). Manish received a Ph.D. in Computer Science from the University of Colorado, Boulder, and a B.Tech. in Mechanical Engineering from the Indian Institute of Technology, Delhi.

Abstract: The state of artificial intelligence technology has a rich history that dates back decades and includes two fall-outs before the explosive resurgence of today, which is credited largely to data-driven learning approaches. While AI technology has and continues to become increasingly mainstream with impact across domains and industries, it's not without several drawbacks, weaknesses, and potential to cause undesired effects. The catalogue of AI approaches is vast with many techniques and variants, but a simple classifier separates approaches that are primarily, and often solely, data-driven (leveraging little to no knowledge), from those that do leverage knowledge. Within the national security domain, purely data-driven learning approaches can particularly lead to serious unwanted consequences, and furthermore, there is ample scientific and domain-specific knowledge that can be leveraged to advance knowledge-informed AI for problems of national importance. This report shares findings from a thorough exploration of AI approaches that exploit both data and knowledge (a.k.a. knowledge-informed AI). Specifically, we review illuminating examples of knowledge-informed deep learning and knowledge-informed reinforcement learning for the performance gains they provide (quantified and qualified) to summarize the current state-of-the-art. We also discuss an apparent trade space across variants of knowledge-informed AI, along with observed and prominent issues that suggest worthwhile future research directions. Most importantly, this report suggests how the advantages of knowledge-informed AI, specifically knowledge-informed deep learning and reinforcement learning, stands to benefit the national security domain.

Bio: Anu Myne is currently serving as an associate to the chief technology officer at MIT Lincoln Laboratory. She received her B.S. in electrical engineering from Worcester Polytechnic Institute, joined the Laboratory in 2006, and pursued her M.S. degree from Northeastern University while working. For several years of her career, she focused heavily in the areas of radar, radar-spoofing, and how to effectively test these technologies. She then shifted focus to applying AI within the same national security problem areas – she developed AI for intelligent test and evaluation using classical machine learning, as well as image classification algorithms using modern AI techniques. She stepped into her current role in 2018 and began focusing on AI and all its applications across the Laboratory, as well as ethical AI, and the vast and growing sub-field of what some refer to as informed AI. From her first time meddling with the more modern, deep learning techniques, Anu thought about how integrating principled knowledge into these learning pipelines could speed up learning, require fewer resources, and lead to breakthroughs faster. Knowledge-integrated informed AI is now an important strategic direction at the Laboratory that she hopes to expand to bring far-reaching impact to the broader set of national security problems and solutions.

Abstract: Electronic structure calculations are used throughout chemistry and materials science to find new drugs and materials. About one-third of US supercomputing power is now spent on this task (more than on climate change). But all such calculations rely on density functional theory, which requires approximating the electronic energy. Today's human-designed functionals yield useful accuracy, but have many limitations and flaws. A race is on to use machine learning to find better density functionals. I will discuss some of the latest results in this quest.

Learning to Approximate Density Functionals Kalita, Bhupalee, Li, Li, McCarty, Ryan J. and Burke, Kieron, Accounts of Chemical Research 54, 818-826 (2021)

Bio: Kieron Burke is a professor in both the chemistry and physics departments at UC Irvine. His research focusses on developing a theory of quantum mechanics called density functional theory. Prof Burke works on developing all aspects of DFT: formalism, extensions to new areas, new approximations, and simplifications. His work is heavily used in materials science, chemistry, matter under extreme conditions (such as planetary interiors or fusion reactors), magnetic materials, molecular electronics, and so on. He has given talks in theoretical chemistry, condensed matter physics, applied mathematics, computer science, and even organic chemistry. Prof Burke is a Distinguished Professor of UC Irvine. He is also a fellow of the American Physical Society, the British Royal Society for Chemistry, and the American Association for the Advancement of Science, and a member of the International Academy of Quantum Molecular Sciences. He is known around the world for his many educational and outreach activities. According to google scholar, his research papers are now cited almost 20,000 times each year.

Abstract: Physics-based models are widely used to simulate water temperature and streamflow. Although they are built based on general physical laws, these models often produce biased simulations due to inaccurate parameterizations or approximations used to represent the true physics. Moreover, existing approaches are not designed for capturing the impact of human infrastructures, such as dams and reservoirs. Machine learning models often achieve better accuracy for well-observed streams and lakes, but perform poorly when adapted to poorly-observed locations. In this presentation, we introduce two related research tasks for advancing data-driven approaches for modeling stream networks by leveraging simulated data produced by physics-based models. First, we build a new data-driven framework to monitor dynamical systems by extracting general scientific knowledge embodied in simulation data generated by the physics-based model. To handle the bias in simulation data caused by imperfect parameterization, we propose to extract general physical relationships jointly from multiple sets of simulations generated by a physics-based model under different physical parameters, and then fine-tune the model using limited observation data via a constrastive learning process. This method not only produce better predictions, but also provide insights about the variation of physical parameters over space and time. Second, we propose a new data-driven method for modeling the impact of reservoirs on stream water temperature. A pseudo-perspective learning method is proposed to mimic the water managers' release decision, which effectively handles reservoirs without available release information. We also extract water flow patterns using physical simulations of reservoirs and transfer such knowledge to guide the learning process for the entire stream networks.

Bio: Xiaowei Jia is an Assistant Professor in the Department of Computer Science at the University of Pittsburgh. He obtained his Ph.D. degree at the University of Minnesota, under the supervision of Prof. Vipin Kumar. Prior to that, he received his M.S. degree from State University of New York at Buffalo and his B.S. degree from University of Science and Technology of China. His research interests include spatio-temporal data mining, physics-guided data science, and deep learning. His research has been published in major journals in data mining (e.g., TKDE) and scientific journals, as well as top-tier conferences (e.g., SIGKDD, ICDM, SDM, and CIKM). Jia was the recipient of UMN Doctoral Dissertation Fellowship (2019), the Best Applied Data Science Paper Award in SDM 21, the Best Conference Paper Award in ASONAM 16, and the Best Student Paper Award in BIBE 14.

Abstract: A multitude of physical systems applications including design, control, diagnosis, prognostics, and host of other problems are predicated on the assumption of model availability. There are mainly two approaches to modeling: Physics/Equation based modeling (Model-Based, MB) and Machine Learning (ML). Model-based methods assume the availability of an accurate system model while data-driven methods are based on machine learning. Purely data-driven ML methods ignore any knowledge about the physical/abstract systems. Additionally, ML approaches require a large amount of labeled training data that is typically unavailable. MB approaches require excellent physics models and good specification of parameters values. When building models of complex systems, we are often limited by the unavailability of the parameters of the system components due to incomplete technical specifications, hidden physical interactions or interactions that are too complex to model from first principles. Hence, we often make simplifying assumptions (e.g., linear approximations) and construct coarse models that imperfectly describe the behavior of the real system. A prudent approach is to use hybrid methods that use the physics of system and prior knowledge about the domain to guide construction of machine learning techniques such as Deep Neural Networks (DNNs). The principal goal of this talk is to discuss challenges related to the development of hybrid methods that combine Multi-physics equation-based models with data-driven machine learning models (such as DNNs) to enable predictive modeling of complex systems in the presence of imperfect models and sparse and noisy data. I will discuss connections to larger problems in the associated area and present specific results related to the development of novel hybrid methods.

Bio: Dr. Rahul Rai joined the Department of Automotive Engineering in 2020 as Dean’s Distinguished Professor in the Clemson University International Center for Automotive Research (CU-ICAR). He directs the Geometric Reasoning and Artificial Intelligence Lab (GRAIL, which is located at both CU-ICAR and Center for Manufacturing Innovation (CMI). Previously, he served on the Mechanical and Aerospace Engineering faculty at the University at Buffalo-SUNY (2012-2020). Dr. Rai also has industrial research center experiences at United Technology Research Center (UTRC) and Palo Alto Research Center (PARC). Dr. Rai received his B.Tech. degree in 2000 and M.S. degree in 2002 in Manufacturing Engineering from the National Institute of Foundry and Forge Technology (NIFFT), Ranchi, India, and Missouri University of Science and Technology (Missouri S&T) USA, respectively. He earned his doctoral degree in Mechanical Engineering from The University of Texas at Austin USA in 2006. Dr. Rai’s research is focused on developing computational tools for Manufacturing, Cyber-Physical System (CPS) Design, Autonomy, Collaborative Human-Technology Systems, Diagnostics and Prognostics, and Extended Reality (XR) domains. By combining engineering innovations with methods from machine learning, AI, statistics and optimization, and geometric reasoning, his research strives to solve important problems in the above-mentioned domains. His research has been supported by NSF, DARPA, ONR, ARL, NSWC, DMDII, CESMII, HP, NYSERDA, and NYSPII (funding totaling more than $20M as PI/Co-PI). He has authored over 100 papers to date in peer-reviewed conferences and journals covering a wide array of problems. Dr. Rai is the recipient of numerous awards, including the 2009 HP Design Innovation, 2017 ASME IDETC/CIE Young Engineer Award, and 2019 PHM society conference best paper award. Additionally, Dr. Rai is Associate Editor of the International Journal of Production Research and ASME Journal of Computing and Information Science in Engineering (JCISE) journals and has taken significant leadership roles within the ASME Computers and Information in Engineering professional society.

Abstract: Extreme events such as floods, droughts and heatwaves are projected to increase due to climate change resulting in impacts to stream water availability and quality. In this talk, I will describe our research activities building machine learning models for predicting stream temperatures to determine the impacts of climatic disturbances. The models need to account for the unpredictable timing, duration and spatial extent of extreme events, by generalizing to unmonitored locations, and predicting temperatures on both short and long timescales. We use low-complexity machine learning models (Support Vector Regression, and XGBoost) and deep learning models (LSTMs) to predict monthly and daily stream water temperature at local to regional scales and incorporate process knowledge from heat budget models for selection of inputs and scaling attributes. We demonstrate the application of these models for the mid-Atlantic and Pacific Northwest hydrological basins with differing climate, geological, land use and water management attributes.

Bio: Charuleka Varadharajan is a Research Scientist at Lawrence Berkeley National Lab. As a biogeochemist and environmental data scientist, she is interested in the water, energy and carbon nexus to understand and limit the impacts of human activities on water resources and climate. Her research has previously involved studying the fate, transport and mitigation of contaminants in groundwater; measurement and prediction of carbon fluxes in terrestrial and subsurface environments; and management, synthesis, and analysis of diverse multi-scale environmental datasets. Her expertise spans various techniques for data collection and analysis, including laboratory experiments; x-ray synchrotron spectroscopy; sensor-based field data collection; web-based software to integrate distributed datasets in real-time; and the use of geoinformatics, statistical, and wavelet-based data processing to analyze high spatial and temporal resolution data. She is currently interested in enhancing LBNL’s environmental data capabilities towards building an environmental knowledgebase, in partnership with the Computational Research Division.

Abstract: A data-driven model can be built to accurately accelerate computationally expensive physical simulations, which is essential in multi-query problems, such as inverse problem, uncertainty quantification, design optimization, and optimal control. In this talk, two types of data-driven model order reduction techniques will be discussed, i.e., the black-box approach that incorporates only data and the physics-constrained approach that incorporates the first principle as well as data. The advantages and disadvantages of each method will be discussed. Several recent developments of generalizable and robust data-driven physics-constrained reduced order models will be demonstrated for various physical simulations as well. For example, a hyper-reduced time-windowing reduced order model overcomes the difficulty of advection-dominated shock propagation phenomenon, achieving a speed-up of O(20~100) with a relative error much less than 1% for Lagrangian hydrodynamics problems. The nonlinear manifold reduced order model also overcomes the challenges posed by the problems with Kolmogorov’s width decaying slowly by representing the solution field with a compact neural network decoder, i.e., nonlinear manifold. The space–time reduced order model accelerates a large-scale particle Boltzmann transport simulation by a factor of 2,700 with a relative error less than 1%. Furthermore, successful application of these reduced order models in design optimization problems will be presented. Finally, the library for reduced order models, i.e., libROM, and its webpage will be introduced, which is useful for educational as well as research purpose.

Bio: Youngsoo Choi is a computational math scientist in CASC under Computing directorate at LLNL. His research focuses on developing efficient reduced order models for various physical simulations for time-sensitive decision-making multi-query problems, such as inverse problems, design optimization, and uncertainty quantification. Together with his collaborators, he has developed various powerful model order reduction techniques, such as machine learning-based nonlinear manifold and space-time reduced order models for nonlinear dynamical systems. He has also developed the component-wise reduced order model optimization algorithm, which enables fast and accurate computational modeling tool for lattice-structure design. He is currently leading data-driven surrogate modeling development team for various physical simulations, with whom he developed the open source codes, libROM and LaghosROM. He is also involved with quantum computing research. He has earned his undergraduate degree for Civil and Environmental Engineering from Cornell University and his PhD degree for Computational and Mathematical Engineering from Stanford University. He was a postdoc at Sandia National Laboratories and Stanford University prior to joining LLNL in 2017.