Project page
"A sparse Bayesian framework for discovering interpretable nonlinear stochastic dynamical systems with Gaussian white noise"
"A sparse Bayesian framework for discovering interpretable nonlinear stochastic dynamical systems with Gaussian white noise"
Extracting governing physics from data is a key challenge in many areas of science and technology. The existing techniques for equation discovery are mostly applicable to deterministic systems and require both input and state measurements. We here propose a novel data-driven framework for discovering nonlinear stochastic dynamical systems excited with Gaussian white noise. The proposed framework blends concepts of stochastic calculus, sparse learning algorithms, and Bayesian statistics to learn the governing physics from data. The proposed framework is highly efficient and works with sparse, noisy, and incomplete output measurements.
Extracting governing physics from data is a key challenge in many areas of science and technology. The existing techniques for equation discovery are mostly applicable to deterministic systems and require both input and state measurements. We here propose a novel data-driven framework for discovering nonlinear stochastic dynamical systems excited with Gaussian white noise. The proposed framework blends concepts of stochastic calculus, sparse learning algorithms, and Bayesian statistics to learn the governing physics from data. The proposed framework is highly efficient and works with sparse, noisy, and incomplete output measurements.
Data-driven Bayesian Discovery of Stochastic Duffing Oscillator
Model Discovery of Stochastic Duffing Oscillator
Model Discovery of Stochastic Duffing Oscillator
Simultaneous Parameter Identification
Simultaneous Parameter Identification
*** The source codes for reproducing the results presented in this paper can be accessed at https://github.com/csccm-iitd/learning-sde-from-data. ***
"Wavelet Neural Operator for solving parametric partial differential equations in computational mechanics problems"
"Wavelet Neural Operator for solving parametric partial differential equations in computational mechanics problems"
Here, we introduce the Wavelet Neural Operator (WNO) for learning the parametric mapping between input-output function spaces. The WNO blends conventional neural network integral kernels with wavelet transformation. WNO harnesses the superiority of the wavelets in space-frequency localization of the functions and enables accurate tracking of patterns in the spatial domain and effective learning of the functional mappings. Since the wavelets are localized in both time/space and frequency, WNO can provide high spatial and frequency resolution. This offers the learning of the finer details of the parametric dependencies in solving complex problems.
Here, we introduce the Wavelet Neural Operator (WNO) for learning the parametric mapping between input-output function spaces. The WNO blends conventional neural network integral kernels with wavelet transformation. WNO harnesses the superiority of the wavelets in space-frequency localization of the functions and enables accurate tracking of patterns in the spatial domain and effective learning of the functional mappings. Since the wavelets are localized in both time/space and frequency, WNO can provide high spatial and frequency resolution. This offers the learning of the finer details of the parametric dependencies in solving complex problems.
Prediction of 24-hour activity of Cyclone Amphan from previous 24 hour wind-velocity data.
Prediction of 24-hour activity of Cyclone Amphan from previous 24 hour wind-velocity data.
*** The source codes for reproducing the results presented in this paper can be accessed at https://github.com/csccm-iitd/WNO. ***
"Physics informed WNO"
"Physics informed WNO"
Deep neural operators are recognized as an effective tool for learning solution operators of complex partial differential equations (PDEs). As compared to laborious analytical and computational tools, a single neural operator can predict solutions of PDEs for varying initial or boundary conditions and different inputs without repeated independent runs from scratch. The Wavelet Neural Operator (WNO) is one such operator that harnesses the advantage of time–frequency localization of wavelets to capture the manifolds in the spatial domain effectively. However, the data-hungry nature of the framework is a major shortcoming. Relying completely on conventional solvers for data generation and subsequently training operators with generated data leads to a time-consuming and challenging implementation of the operators in practical applications. We propose a physics-informed WNO for learning the solution operators of families of parametric PDEs without labeled training data.
Deep neural operators are recognized as an effective tool for learning solution operators of complex partial differential equations (PDEs). As compared to laborious analytical and computational tools, a single neural operator can predict solutions of PDEs for varying initial or boundary conditions and different inputs without repeated independent runs from scratch. The Wavelet Neural Operator (WNO) is one such operator that harnesses the advantage of time–frequency localization of wavelets to capture the manifolds in the spatial domain effectively. However, the data-hungry nature of the framework is a major shortcoming. Relying completely on conventional solvers for data generation and subsequently training operators with generated data leads to a time-consuming and challenging implementation of the operators in practical applications. We propose a physics-informed WNO for learning the solution operators of families of parametric PDEs without labeled training data.
*** The source codes for reproducing the results presented in this paper can be accessed at https://github.com/csccm-iitd/Physics-informed-WNO. ***
"Robust model agnostic predictive control algorithm for randomly excited dynamical systems"
"Robust model agnostic predictive control algorithm for randomly excited dynamical systems"
We propose a novel robust model agnostic predictive control (RoMAn-MPC) algorithm and illustrate its application in randomly excited dynamical systems. Unlike conventional model predictive control algorithms, the proposed RoMAn-MPC requires no information about the underlying physics of the system; instead, the governing physics is identified by using the recently proposed stochastic equation discovery framework. One key advantage of the proposed approach resides in its capability to generalize; this eliminates the repeated retraining phase — a major bottleneck with other machine learning-based model agnostic control algorithms. Overall, the proposed RoMAn-MPC (a) is robust against measurement noise, (b) works with sparse measurements, (c) can tackle set-point changes, (d) works with multiple control variables, and (e) can incorporate dead time.
We propose a novel robust model agnostic predictive control (RoMAn-MPC) algorithm and illustrate its application in randomly excited dynamical systems. Unlike conventional model predictive control algorithms, the proposed RoMAn-MPC requires no information about the underlying physics of the system; instead, the governing physics is identified by using the recently proposed stochastic equation discovery framework. One key advantage of the proposed approach resides in its capability to generalize; this eliminates the repeated retraining phase — a major bottleneck with other machine learning-based model agnostic control algorithms. Overall, the proposed RoMAn-MPC (a) is robust against measurement noise, (b) works with sparse measurements, (c) can tackle set-point changes, (d) works with multiple control variables, and (e) can incorporate dead time.
*** The source codes for reproducing the results presented in this paper can be accessed at https://github.com/csccm-iitd/RoMAn. ***
"Discovering stochastic partial differential equations from limited data using variational Bayes inference"
"Discovering stochastic partial differential equations from limited data using variational Bayes inference"
This work proposes a novel framework for discovering Stochastic Partial Differential Equations (SPDEs) from data. The proposed approach combines the concepts of stochastic calculus, variational Bayes theory, and sparse learning. We propose the extended Kramers–Moyal expansion to express the drift and diffusion terms of an SPDE in terms of state responses and use Spike-and-Slab priors with sparse learning techniques to efficiently and accurately discover the underlying SPDEs. The proposed approach can accurately identify the underlying SPDEs with limited data. This is the first attempt at discovering SPDEs from data, and it has significant implications for various scientific applications, such as climate modeling, financial forecasting, chemical kinetics, etc. where the knowledge of external disturbances is limited.
This work proposes a novel framework for discovering Stochastic Partial Differential Equations (SPDEs) from data. The proposed approach combines the concepts of stochastic calculus, variational Bayes theory, and sparse learning. We propose the extended Kramers–Moyal expansion to express the drift and diffusion terms of an SPDE in terms of state responses and use Spike-and-Slab priors with sparse learning techniques to efficiently and accurately discover the underlying SPDEs. The proposed approach can accurately identify the underlying SPDEs with limited data. This is the first attempt at discovering SPDEs from data, and it has significant implications for various scientific applications, such as climate modeling, financial forecasting, chemical kinetics, etc. where the knowledge of external disturbances is limited.
*** The source codes for reproducing the results presented in this paper can be accessed at https://github.com/csccm-iitd/Stochastic-PDE-Discovery. ***
"Discovering interpretable Lagrangian of dynamical systems from data"
"Discovering interpretable Lagrangian of dynamical systems from data"
A complete understanding of physical systems requires models that are accurate and obey natural conservation laws. Recent trends in representation learning involve learning Lagrangian from data rather than the direct discovery of governing equations of motion. The generalization of equation discovery techniques has huge potential; however, existing Lagrangian discovery frameworks are black-box in nature. This raises a concern about the reusability of the discovered Lagrangian. In this article, we propose a novel data-driven machine-learning algorithm to automate the discovery of interpretable Lagrangian from data. The framework also allows the automated discovery of conservation laws and governing equations of motion, thereby can be considered as a superset of the existing equation discovery techniques.
A complete understanding of physical systems requires models that are accurate and obey natural conservation laws. Recent trends in representation learning involve learning Lagrangian from data rather than the direct discovery of governing equations of motion. The generalization of equation discovery techniques has huge potential; however, existing Lagrangian discovery frameworks are black-box in nature. This raises a concern about the reusability of the discovered Lagrangian. In this article, we propose a novel data-driven machine-learning algorithm to automate the discovery of interpretable Lagrangian from data. The framework also allows the automated discovery of conservation laws and governing equations of motion, thereby can be considered as a superset of the existing equation discovery techniques.
*** The source codes for reproducing the results presented in this paper can be accessed at https://github.com/csccm-iitd/Lagrange-Discovery. ***
"A Bayesian framework for discovering interpretable Lagrangian of dynamical systems from data"
"A Bayesian framework for discovering interpretable Lagrangian of dynamical systems from data"
While the previous method automates the discovery of interpretable Lagrangian at a highly efficient computational cost, the method cannot quantify the model-form uncertainties due to noisy and limited measurements. Thus, we also propose an alternative Bayesian framework for discovering interpretable Lagrangian of dynamical systems from data. At an increased computational cost, the Bayesian framework, in addition to the automated discovery of interpretable Lagrangian, Hamiltonian, and governing differential equations, also quantifies the epistemic uncertainty due to limited data.
While the previous method automates the discovery of interpretable Lagrangian at a highly efficient computational cost, the method cannot quantify the model-form uncertainties due to noisy and limited measurements. Thus, we also propose an alternative Bayesian framework for discovering interpretable Lagrangian of dynamical systems from data. At an increased computational cost, the Bayesian framework, in addition to the automated discovery of interpretable Lagrangian, Hamiltonian, and governing differential equations, also quantifies the epistemic uncertainty due to limited data.
*** The source codes for reproducing the results presented in this paper will be made available publicly upon acceptance ***
"A foundational neural operator that continuously learns without forgetting"
"A foundational neural operator that continuously learns without forgetting"
Machine learning has witnessed substantial growth, leading to the development of advanced artificial intelligence models crafted to address a wide range of real-world challenges spanning various domains, such as computer vision, natural language processing, and scientific computing. Nevertheless, the creation of custom models for each new task remains a resource-intensive undertaking, demanding considerable computational time and memory resources. In this study, we introduce the concept of the Neural Combinatorial Wavelet Neural Operator (NCWNO) as a foundational model for scientific computing. This model is specifically designed to excel in learning from a diverse spectrum of physics and continuously adapt to the solution operators associated with parametric partial differential equations (PDEs). The NCWNO leverages a gated structure that employs local wavelet experts to acquire shared features across multiple physical systems, complemented by a memory-based ensembling approach among these local wavelet experts. This combination enables rapid adaptation to new challenges. The proposed foundational model offers two key advantages: (i) it can simultaneously learn solution operators for multiple parametric PDEs, and (ii) it can swiftly generalize to new parametric PDEs with minimal fine-tuning. The proposed NCWNO is the first foundational operator learning algorithm distinguished by its (i) robustness against catastrophic forgetting, (ii) the maintenance of positive transfer for new parametric PDEs, and (iii) the facilitation of knowledge transfer across dissimilar tasks.
Machine learning has witnessed substantial growth, leading to the development of advanced artificial intelligence models crafted to address a wide range of real-world challenges spanning various domains, such as computer vision, natural language processing, and scientific computing. Nevertheless, the creation of custom models for each new task remains a resource-intensive undertaking, demanding considerable computational time and memory resources. In this study, we introduce the concept of the Neural Combinatorial Wavelet Neural Operator (NCWNO) as a foundational model for scientific computing. This model is specifically designed to excel in learning from a diverse spectrum of physics and continuously adapt to the solution operators associated with parametric partial differential equations (PDEs). The NCWNO leverages a gated structure that employs local wavelet experts to acquire shared features across multiple physical systems, complemented by a memory-based ensembling approach among these local wavelet experts. This combination enables rapid adaptation to new challenges. The proposed foundational model offers two key advantages: (i) it can simultaneously learn solution operators for multiple parametric PDEs, and (ii) it can swiftly generalize to new parametric PDEs with minimal fine-tuning. The proposed NCWNO is the first foundational operator learning algorithm distinguished by its (i) robustness against catastrophic forgetting, (ii) the maintenance of positive transfer for new parametric PDEs, and (iii) the facilitation of knowledge transfer across dissimilar tasks.
*** The source codes for reproducing the results presented in this paper will be made available publicly upon acceptance ***
"A wavelet neural operator based elastography for localization and quantification of tumors"
"A wavelet neural operator based elastography for localization and quantification of tumors"
Elastography is an imaging modality where an inverse problem is solved to extract the elastic properties of tissues and subsequently mapped to anatomical images for diagnostic purposes. In the present work, we propose a wavelet neural operator-based approach for correctly learning the non-linear mapping of elastic properties directly from measured displacement field data. The proposed framework learns the underlying operator behind the elastic mapping and thus can map any displacement data from a family to the elastic properties. The displacement fields are first uplifted to a high-dimensional space using a fully connected neural network. On the lifted data, certain iterations are performed using wavelet neural blocks. The mapping between the displacement and the elasticity using wavelets is unique and remains stable during training. The proposed framework is tested on several artificially fabricated numerical examples, including a benign-cum-malignant tumor prediction problem.
Elastography is an imaging modality where an inverse problem is solved to extract the elastic properties of tissues and subsequently mapped to anatomical images for diagnostic purposes. In the present work, we propose a wavelet neural operator-based approach for correctly learning the non-linear mapping of elastic properties directly from measured displacement field data. The proposed framework learns the underlying operator behind the elastic mapping and thus can map any displacement data from a family to the elastic properties. The displacement fields are first uplifted to a high-dimensional space using a fully connected neural network. On the lifted data, certain iterations are performed using wavelet neural blocks. The mapping between the displacement and the elasticity using wavelets is unique and remains stable during training. The proposed framework is tested on several artificially fabricated numerical examples, including a benign-cum-malignant tumor prediction problem.
*** The source codes for reproducing the results presented in this paper can be accessed at https://www.github.com/csccm-iitd/WNO-elastography. ***
"Probabilistic machine learning based predictive and interpretable digital twin for dynamical systems"
"Probabilistic machine learning based predictive and interpretable digital twin for dynamical systems"
A framework for creating and updating digital twins for dynamical systems from a library of physics-based functions is proposed. The sparse Bayesian machine learning is used to update and derive an interpretable expression for the digital twin. Two approaches for updating the digital twin are proposed. The first approach makes use of both the input and output information from a dynamical system, whereas the second approach utilizes output-only observations to update the digital twin. Both methods use a library of candidate functions representing certain physics to infer new perturbation terms in the existing digital twin model. In both cases, the resulting expressions of updated digital twins are identical, and in addition, the epistemic uncertainties are quantified.
A framework for creating and updating digital twins for dynamical systems from a library of physics-based functions is proposed. The sparse Bayesian machine learning is used to update and derive an interpretable expression for the digital twin. Two approaches for updating the digital twin are proposed. The first approach makes use of both the input and output information from a dynamical system, whereas the second approach utilizes output-only observations to update the digital twin. Both methods use a library of candidate functions representing certain physics to infer new perturbation terms in the existing digital twin model. In both cases, the resulting expressions of updated digital twins are identical, and in addition, the epistemic uncertainties are quantified.
*** The source codes for reproducing the results presented in this paper can be accessed at https://github.com/csccm-iitd/Interpretable-DT. ***
"MAntRA: A framework for model agnostic reliability analysis"
"MAntRA: A framework for model agnostic reliability analysis"
A novel model-agnostic data-driven reliability analysis framework for time-dependent reliability analysis. The proposed approach – referred to as MAntRA – combines Bayesian inference and stochastic differential equations to evaluate the reliability of stochastically-driven dynamical systems for which the governing physics is a priori unknown. A two-stage approach is adopted: in the first stage, an efficient variational Bayesian equation discovery algorithm is developed to determine the governing physics of an underlying stochastic differential equation (SDE) from measured output-only data. The developed algorithm is efficient and accounts for epistemic uncertainty due to limited and noisy data and aleatoric uncertainty because of environmental effects and external excitation. In the second stage, the discovered SDE is solved using a stochastic integration scheme, and the probability of failure is computed.
A novel model-agnostic data-driven reliability analysis framework for time-dependent reliability analysis. The proposed approach – referred to as MAntRA – combines Bayesian inference and stochastic differential equations to evaluate the reliability of stochastically-driven dynamical systems for which the governing physics is a priori unknown. A two-stage approach is adopted: in the first stage, an efficient variational Bayesian equation discovery algorithm is developed to determine the governing physics of an underlying stochastic differential equation (SDE) from measured output-only data. The developed algorithm is efficient and accounts for epistemic uncertainty due to limited and noisy data and aleatoric uncertainty because of environmental effects and external excitation. In the second stage, the discovered SDE is solved using a stochastic integration scheme, and the probability of failure is computed.
*** The source codes for reproducing the results presented in this paper can be accessed at https://github.com/csccm-iitd/MAntRA. ***
"Multi-fidelity wavelet neural operator with application to uncertainty quantification"
"Multi-fidelity wavelet neural operator with application to uncertainty quantification"
In this work, we have developed a new framework based on the wavelet neural operator (WNO) for learning the input-output mapping from multi-fidelity datasets. Neural operators require an extensive amount of data for successful training, which is often too expensive. This work alleviates the requirement of extensive amount of data by making use of multi-fidelity learning, where we train the model by making use of a large amount of inexpensive low-fidelity data along with a small amount of expensive high-fidelity data.
In this work, we have developed a new framework based on the wavelet neural operator (WNO) for learning the input-output mapping from multi-fidelity datasets. Neural operators require an extensive amount of data for successful training, which is often too expensive. This work alleviates the requirement of extensive amount of data by making use of multi-fidelity learning, where we train the model by making use of a large amount of inexpensive low-fidelity data along with a small amount of expensive high-fidelity data.
*** The source codes for reproducing the results presented in this paper will be made available publicly upon acceptance ***