Despite recent advances in therapeutic strategies, many cancers are not responsive to treatment. One way in which cancer is consistently able to evade an immune response is through a series of immune checkpoints, or inhibitors. Immune checkpoint blockade seeks to prevent this evasion by competitively binding to the inhibitory receptors on the immune cell surface, thereby reducing the cancer’s opportunities for immune downregulation. We explore the optimization of immune checkpoint blockade from a multiscale perspective. We first consider an ODE system for tumor-immune cellular dynamics and immune checkpoint perturbation at the whole-tumor level. We then consider an agent-based model for tumor-immune interactions at the individual cell level. Finally, we demonstrate the role of immune checkpoint blockade efficiency in our model system. This work holds promise for informing optimal therapeutic design strategies.

Seminar date and time: May 9, 2025. 10 AM ET.

Zoom recording here.

Despite the fact that Parallel-in-Time (PinT) methods are predicted to become necessary to fully utilize next-generation exascale machines, there are currently no known practical methods which scale well with the length of the time-domain for chaotic problems, due to exponential dependence of the condition number on the fastest chaotic time-scale. While most prior works applying multigrid-in-time to chaotic systems focus on the coarse grid equation, we investigate the relaxation techniques commonly used and demonstrate that they in fact diverge for chaotic systems. Here the novel Local Shadowing Relaxation (LSR) is presented and proven to be a convergent, PinT relaxation for chaotic PDE systems. Promising preliminary analytical results and numerical experiments with the Lorenz system indicate that LSR may solve the scaling problem for chaotic systems, potentially allowing space-time parallelization of turbulent computational fluid dynamics.

Seminar date and time: May 2. 10 AM ET.

Zoom recording here.

Nonlinear dynamical systems are ubiquitous in science and engineering. While high-fidelity models based on first principle has been well established, reduced-complexity models are of particular interest in recent years due to its feasibility for many-query tasks, e.g., uncertainty propagation, optimal control and design. In this talk, I will discuss two recent works from my group at RPI in this theme. For the first half, I will talk about learning reduced complexity models in the light of learning Koopman operator for control. We highlight our solutions to address several issues when standard algorithm is applied, e.g., noisy data, instability, efficient curation of training data, choice of observables. We demonstrate the benefits of our proposed framework on model predictive control of a Koopman-based surrogate model for CartPole problem. For the second half, I will talk about learning reduced complexity models in the light of surrogate model of time-dependent partial differential equations from data. In contrast to existing frameworks, our model ensures stability of learned surrogate model through a stable parametrization of Koopman operator and trapping theorem of linear quadratic dynamics. Moreover, our model is agnostic to mesh, scalable to 3D problems and could only require a few sensor measurement during inference stage. We demonstrate the benefits of our model over several state-of-the-art neural operator frameworks on 2D wave propagation, 2D Navier-Stokes in a periodic box, shallow water equations.

Seminar date and time: April 25, 2025. 10 AM ET.

Zoom recording here.

Scientific machine learning (SciML) relies heavily on automatic differentiation (AD), the process of constructing gradients which include machine learning integrated into mechanistic models for the purpose of gradient-based optimization. While these differentiable programming approaches pitch an idea of "simply put the simulator into a loss function and use AD", it turns out there are a lot more subtle details to consider in practice. In this talk we will dive into the numerical analysis of differentiable simulation and ask the question: how numerically stable and robust is AD? We will use examples from the Python-based Jax and PyTorch libraries in order to demonstrate how canonical formulations of AD and adjoint methods can give inaccurate gradients in the context of ODEs and PDEs. We demonstrate cases where the methodologies are "mathematically correct", but due to the intricacies of numerical error propagation, their approaches can give 60% and greater error even in simple cases like linear ODEs.  We'll then describe some of the non-standard modifications to AD which are done in the Julia SciML libraries to overcome these numerical instabilities, crucially also describing the engineering trade-offs which are required to be made in the process. The audience should leave with a greater appreciation of the greater numerical challenges which still need to be addressed in the field of AD for SciML.

Seminar date and time: April 18, 2025. 10 AM ET.

Zoom recording here.

The optimal Petrov-Galerkin formulation to solve PDEs was first introduced in the 1980’s. Its goal was, given a trial basis, to approximate the finite-dimensional solution that is optimal with respect to a suitable norm. The theory is elegant and allows for the recovery of optimal convergence rates, even for problems where standard Galerkin methods fail. However, recovery of the optimal solution is contingent on being able to construct the optimal weighting functions associated with the trial basis. While explicit constructions are available for simple 1D and 2D problems, such constructions for a general multidimensional problem remain elusive. As a result, interest in this approach waned with the rise of alternative finite element strategies. 

In this talk, we revisit optimal weighting functions through the lens of deep learning. We propose an operator network framework called PG-VarMiON, that emulates the optimal Petrov-Galerkin weak form of the underlying PDE. Given a suitable trial basis and a norm, the PG-VarMiON approximates the optimal finite-dimensional while implicitly learning the optimal weighting functions. We derive an approximation error estimate for PG-VarMiON, highlighting the contributions of various error sources. Several numerical results for the advection-diffusion equation are presented to demonstrate the efficacy of the proposed method. By embedding the Petrov-Galerkin structure into the network architecture, PG- VarMiON exhibits greater robustness and improved generalization compared to other deep operator frameworks, particularly when training data is limited. The proposed approach harnesses the knowledge of traditional numerical methods to solve PDEs, paving the way for constructing mathematical-sound and efficient deep surrogates.

Seminar date and time: April 11, 2025. 11 AM ET (note delayed start!)

Zoom recording here (starts a bit late because Romit forgot to hit record -sorry)

Reliable forecasts of the Earth system are crucial for human progress and safety from natural disasters. Artificial intelligence offers substantial potential to improve prediction accuracy and computational efficiency in this field, however this remains underexplored in many domains. Here we introduce Aurora, a large-scale foundation model for the Earth system trained on over a million hours of diverse data. Aurora outperforms operational forecasts for air quality, ocean waves, tropical cyclone tracks, and high-resolution weather forecasting at orders of magnitude smaller computational expense than dedicated existing systems. With the ability to fine-tune Aurora to diverse application domains at only modest computational cost, Aurora represents significant progress in making actionable Earth system predictions accessible to anyone.

Seminar date and time: April 7, 2025. 11 AM ET.

Zoom recording here.

Adjoint methods are powerful engineering design tools for computational fluid dynamics. These methods efficiently compute the sensitivities of an objective function with respect to many input parameters. Adjoint methods have been successfully used for design optimization, error-estimation, flow control, and uncertainty quantification of steady flow simulations such as Reynolds-averaged Navier-Stokes (RANS) solvers. Recently, engineers have started to use scale-resolving simulations including Detached-eddy simulations (DES) and Large-eddy simulations (LES) for flows with unsteady separation and jets. DES and LES model these flows more accurately than RANS and capture the chaotic dynamics inherent in turbulence, making the application of adjoint methods challenging. One of the main challenges is the exponential growth in magnitude of adjoint variables with time in a chaotic dynamical system [1]. This exponential growth causes the conventional adjoint method to compute large, unusable sensitivities for scale-resolving simulations. 

This talk discusses the application of adjoint methods to chaotic systems, especially scale-resolved turbulent flow simulations, using several shadowing methods including non-intrusive least-squares shadowing (NILSS) [2]. NILSS is formulated so that the exponential growth observed in conventional adjoint approaches is controlled and a bounded, useful adjoint is computed. Adjoint computations of time-averaged objective sensitivities, and some preliminary work on dual-weighted residual error estimates will be presented for the minimal flow unit for near-wall turbulence, a channel flow through a truncated domain. This talk concludes with some thoughts on the current state of research on chaotic sensitivity analysis and the future of shadowing-based sensitivity analysis.

[1] Q. Wang, R. Hui, and P. Blonigan. Least Squares Shadowing sensitivity analysis of chaotic limit cycle oscillations. Journal of Computational Physics, 267:210–224, June 2014

[2] P. Blonigan. Adjoint sensitivity analysis of chaotic dynamical systems with non-intrusive least squares shadowing. Journal of Computational Physics, 348:803–826, Nov. 2017

Seminar date and time: March 28, 2025. 10 AM ET.

Zoom recording here.

This talk focuses on how scientific machine learning (SciML) can be effectively applied to develop digital twin technology for nuclear reactors. A key component of digital twin implementation is achieving accurate and efficient computational fluid dynamics (CFD). Recent research highlights the need for SciML techniques that can simultaneously satisfy the following key objectives: accurate prediction of unseen (future) time series in long-term simulations, accelerated computation, and a manageable amount of training data and time. To address these challenges, we propose a three-level SciML application framework: (1) real-time PDE solvers for low-fidelity CFD, (2) hybrid CFD solvers for high-fidelity simulations, and (3) effective flow control for optimal-fidelity CFD. Advanced physics-informed neural networks (PINNs) and deep operator networks (DeepONets) are highlighted as strong candidates for real-time PDE solvers. The residual-based physics-informed transfer learning (RePIT) strategy will be introduced as a hybrid solver. Lastly, this talk will cover inductive-biased deep reinforcement learning to exploit the full potential of the constructed digital twin.

Seminar date and time: March 21, 2025. 10 AM ET.

Zoom recording here.

Physics-Informed Neural Networks (PINNs) provide a flexible, mesh-free approach to solving differential equations. While their implementation is straightforward and they hold great potential for high-dimensional, inverse, and nonlinear problems, training remains challenging due to their sensitivity to weight initialization, hyperparameter selection, scalability issues, spectral bias, and ill-conditioning. This talk explores how overlapping domain decomposition (DD) techniques can improve the convergence and efficiency of PINNs.

Different strategies for integrating DD with PINNs are investigated. First, classical PINNs are enhanced using multilevel DD-based architectures to improve performance. Additionally, this strategy is combined with multifidelity stacking PINNs for time-dependent problems, demonstrating clear improvements over reference results without DD. Second, randomized neural networks are considered, where hidden layer weights are randomly initialized and fixed, reducing the problem to a linear least-squares formulation for linear differential operators. In this setting, DD-based architectures, combined with overlapping Schwarz preconditioning, accelerate convergence. Finally, improvements in physics-informed neural operators through DD-based architectures are explored, enabling the efficient approximation of solution operators for parameterized problems.

Numerical experiments on multiscale and wave phenomena demonstrate the effectiveness of DD techniques in PINNs. These results highlight the potential of DD methods to significantly enhance computational efficiency and accuracy in physics-informed machine learning.

Seminar date and time: March 14, 2025. 10 AM ET.

Zoom recording here.

This is a special hybrid seminar with Professor Chakraborty presenting in-person at E339 Westgate Building, Pennsylvania State University, University Park.

Seminar date and time: March 10, 2025. 10.30 AM ET.

Zoom recording here.

The fabric of our daily lives, from weather forecasts to stock market predictions, from the aerodynamics of vehicles to the development of innovative materials, and even in the realms of medicine and space exploration, relies heavily on scientific and engineering computing. While super-intelligence in AI has made significant strides in language processing, visual recognition, and audio analysis, its potential in the vast domain of natural sciences and engineering remains largely untapped. In this talk, we delve into the evolution of AI from neural networks to neural operators, unlocking new frontiers in advanced scientific computing. Join this talk as we explore how these cutting-edge technologies are revolutionizing our approach to understanding and modeling the complexities of the natural world, paving the way for groundbreaking discoveries and innovations.

Seminar date and time: March 7, 2025. 10 AM ET.

Zoom recording here (part 1) and here (part 2).

Computational science is at the forefront of modern technology, enabling groundbreaking simulations—from quantum molecular dynamics and magnetic fusion to complex wave phenomena. These advances have ushered in the era of digital twins, accelerating the design, build, test, and learn (DBTL) process across diverse applications. Yet even the most advanced simulations remain computationally expensive, often pushing high-performance computing systems to their limits. Fortunately, machine learning (ML) and artificial intelligence (AI) offer promising strategies to enhance simulation speed without sacrificing accuracy. In this talk, I will introduce several foundation models designed to accelerate computational simulations. I will critically evaluate these models, distinguishing between those that offer nice, yet incomplete gains and those that deliver genuine, comprehensive performance improvements. A highlight of the discussion will be the data-driven finite element method (DD-FEM), a robust foundation model whose effectiveness I will demonstrate across three applications: lattice-type structure design, steady Navier–Stokes porous media flow, and time-dependent 2D Burgers advective flow with multiple disturbances, achieving roughly 1000x speed-up and 100x scale-up with a relative error of O(1%).

Seminar date and time: February 28, 2025. 10 AM ET.

Zoom recording here.

The development of reduced-order models has gained significant traction in the field of fluid mechanics. These models offer a means to represent complex systems within an optimally reduced-dimensional space, enabling the creation of predictive models with lower computational costs and facilitating the discovery of underlying dynamical insights. Reduced-order models can be constructed using either a physics-based approach, which leverages governing equations, or data-driven methods, which utilize available training data for the system. These models are particularly well-suited for control applications, where their reduced complexity allows for real-time implementation and efficient control strategy development. This presentation explores the application of both data-driven and physics-based modeling techniques to effectively capture and predict complex fluid dynamical behaviors, with particular emphasis on challenges pertinent to engineering applications, especially in the realm of unsteady aerodynamics. 

Seminar date and time: February 21, 2025. 10 AM ET.

Zoom recording here.

Machine learning (ML) methods have garnered significant interest as potential methods for learning surrogate models for complex engineering systems for which traditional simulation is expensive. However, in many scientific and engineering settings, training data are scarce due to the cost of generating data from traditional high-fidelity simulations. ML models trained on scarce data have high variance and are sensitive to vagaries of the training data set. We propose a new multifidelity training approach for scientific machine learning that exploits the scientific context where data of varying fidelities and costs are available; for example high-fidelity data may be generated by an expensive fully resolved physics simulation whereas lower-fidelity data may arise from a cheaper model based on simplifying assumptions. We use the multifidelity data to define new multifidelity Monte Carlo estimators for the unknown parameters of linear regression models, and provide theoretical analyses that guarantee accuracy and improved robustness to small training budgets. Numerical results show that multifidelity learned models achieve order-of-magnitude lower expected error than standard training approaches when high-fidelity data are scarce.

Seminar date and time: February 14, 2025. 10 AM ET.

Zoom recording here.

Physics-Informed Neural Networks (PINNs) are increasingly used to solve various partial differential equations (PDEs), especially in high dimensions. In real-world applications, data samples are noisy, making it essential to understand the conditions under which a predictor can achieve a small empirical risk. In this work, we present a first-of-its-kind lower bound on the size of neural networks required for the supervised PINN empirical risk to fall below the variance of noisy supervision labels. Specifically, we show that to achieve low training error on Ns data points, the number of parameters d_N must satisfy d_N log d_N ≳ N_s. We show that a similar constraint applies to the fully unsupervised PINN setting when boundary labels are sampled noisily. Consequently, increasing the number of noisy supervision labels alone does not provide a “free lunch” in reducing empirical risk. We investigate PINNs applied to the Hamilton–Jacobi–Bellman (HJB) PDE as a case study to understand the role of the number of trainable parameters in minimizing the associated PINN empirical risk. Our findings lay the groundwork for a program on rigorously quantifying parameter requirements for effective PINN training under noisy conditions. This work was done in collaboration with Dr. Anirbit Mukherjee, University of Mancheseter and Dr. Matthew Colbrook, DAMTP, University of Cambridge

Seminar date and time: February 7, 2025. 10 AM ET.

Zoom recording here.

Scientific machine learning (SciML) has emerged as a promising field for modeling complex fluid flows, particularly those encountered in advanced power generation and propulsion concepts. However, these and other aerospace applications impose stringent requirements on SciML models, necessitating solutions that address both complex physics and geometry. To ensure the effectiveness of new models for power and propulsion applications, machine learning methods must be (a) physically interpretable, (b) inherently compatible with complex data representations, and (c) scalable. In this talk, we introduce new SciML models for fluid flows based on interpretable geometric deep learning to address these requirements, and discuss how hardware-oriented machine learning strategies can enable transformative advancements in simulation capabilities for high-speed reacting flows.

Seminar date and time: January 24, 2025. 10 AM ET.

Zoom recording here.

In this talk, we discuss how highly unsteady flows can be analyzed in a data-driven manner from the aspect of global field reconstruction, reduced-complexity modeling, and flow control.  In particular, our focus is on flows exhibiting unsteadiness with larger amplitudes than those often examined in traditional fluid dynamics.  We first perform global field reconstruction from sparse sensors through the lens of generalized super-resolution analysis. This talk covers not only fundamental applications of fluid-flow super resolution but also practical uses for moving sensor conditions and industrial turbulence. To perform flow control leveraging the reconstructed fields from sparse sensors, we then aim to construct a control strategy of flows in a low-order subspace identified by nonlinear machine-learning-based data compression. Although it is generally challenging to analyze the nonlinear, transient nature of highly unsteady flows with conventional linear techniques, we reveal that the underlying physics of a collection of time-varying vortical flows in a high-dimensional space can be expressed on a low-rank manifold leveraging the present data-driven compression. It is also demonstrated that efficient control strategies can be derived at a minimal cost with the assistance of phase-amplitude reduction on the discovered manifold. Toward the end of the talk, based on the current findings, we discuss how we should be mindful of preparing training data of turbulent flows for data-driven studies in a smart manner.

Seminar date and time: January 17, 2025. 10 AM ET.

Zoom recording available here.

Modern machine learning has shown remarkable promise in multiple applications. However, brute force use of neural networks, even when they have huge numbers of trainable parameters, can fail to provide highly accurate predictions for problems in the physical sciences. We present a collection of ideas about how enforcing physics, exploiting multifidelity knowledge and the kernel representation of neural networks can lead to significant increase in efficiency and/or accuracy. Various examples are used to illustrate the ideas.

Seminar date and time: December 13, 2024. 10 AM ET.

Recording available here.

Data-driven forecasting of collective dynamics is a challenging problem because short- and long-range interactions combine in complex ways to influence global system properties. Graph neural networks (GNNs) are a powerful class of methods that can be used for modeling collective motion, but GNNs struggle when graphs become too large and their effectiveness at capturing the aforementioned global properties remains unclear. Here we show that by constructing hierarchical and equivariant GNNs, we accurately predict local and global behavior in systems with collective motion. In particular, we validate this approach on simulated datasets of clusters of point vortices and of microswimmers. For the point vortices, we define a local graph of vortices within a cluster and a global graph of interactions between clusters. This approach drastically improves the tracking capabilities over a fully connected graph. Then, by incorporating equivariance to rotations and translations, this method successfully conserves the Hamiltonian for long periods. For the microswimmers, we define a local graph around each microswimmer and a global graph that groups the long-range interactions of many microswimmers. We also find that this approach – along with accounting for equivariance – allows us to both accurately predict short-time dynamics, and predict long-time statistics. Notably, this method effectively predicts the phase transition from aggregation to swirling in this microswimmer system.

Seminar date and time: December 6, 2024. 10 AM ET.

Zoom recording here.

The widespread adoption of data-driven modeling tools and advances in hardware architectures across the scientific computing community have also shaped the future of CFD solvers. In this context, we will review the opportunities and challenges arising from the use of ML models in scale-resolving turbulent flow simulations. As examples, the cases of data-driven turbulence modelling, and active flow control using reinforcement learning will be discussed. We will also present our Julia GPU-accelerated CFD solver, WaterLily, and discuss future research directions.

Seminar date and time: November 29, 2024. 10 AM ET.

See recording here

We highlight the risks of the current industrial AI practices involving training large-scale generative models on vast amounts of data scraped from the internet. This process unwittingly leads to training newer models on increasing amounts of AI-synthesized data that is rapidly proliferating online, a phenomenon we refer to as ``model autophagy'' (self-consuming models). We show that without a sufficient influx of fresh, real data at each stage of an autophagous loop, future generative models will inevitably suffer a decline in either quality (precision) or diversity (recall). To mitigate this issue and inspired by fixed-point optimization, we introduce a penalty to the loss function of generative models that minimizes discrepancies between the model's weights when trained on real versus synthetic data. Since computing this penalty would require training a new generative model at each iteration, we propose a permutation-invariant hypernetwork to make evaluating the penalty tractable by dynamically mapping data batches to model weights. This ensures scalability and seamless integration of the penalty term into existing generative modeling paradigms, mitigating biases associated with model autophagy. Additionally, this penalty improves the representation of minority classes in imbalanced datasets, which is a key step toward enhancing fairness in generative models.

Seminar date and time: November 22, 2024. 10 AM ET.

Recording available here

We integrate neural operators with diffusion models to address the spectral limitations of neural operators in surrogate modeling of turbulent flows. While neural operators offer computational efficiency, they exhibit deficiencies in capturing high-frequency flow dynamics, resulting in overly smooth approximations. To overcome this, we condition diffusion models on neural operators to enhance the resolution of turbulent structures. Our approach is validated for different neural operators on diverse datasets, including a high Reynolds number jet flow simulation and experimental Schlieren velocimetry. The proposed method significantly improves the alignment of predicted energy spectra with true distributions compared to neural operators alone. Additionally, proper orthogonal decomposition analysis demonstrates enhanced spectral fidelity in space-time. This work establishes a new paradigm for combining generative models with neural operators to advance surrogate modeling of turbulent systems, and it can be used in other scientific applications that involve microstructure and high-frequency content.

See our project page: https://vivekoommen.github.io/NO_DM/

Seminar date and time: November 15, 2024. 10 AM ET.

Seminar recording here.

Deep operator network (DeepONet) has demonstrated significant potential in solving partial differential equations (PDEs) by leveraging neural networks to learn mappings between function spaces. However, their performance deteriorates as the system size and complexity increase. Recent advancements with Latent DeepONet have shown promise in accelerating surrogate models for these complex systems by learning operators in low-dimensional latent spaces. Despite their potential, the Latent DeepONet architectures rely exclusively on data-driven training, necessitating large datasets and proving unsuitable for physics-informed training. To address these limitations, we introduce latent operator learning in a physics-informed framework, termed PI-Latent-DeepONet. Our method employs a two-stacked DeepONet framework: the first DeepONet learns the latent representations, while the second DeepONet recovers the solution in the original space. We demonstrate the effectiveness of the proposed framework in rapidly predicting solutions for high-dimensional PDEs.

Seminar date and time: November 8, 2024. 10 AM ET.

Recording available here.

In this presentation we first use a framework for deep-learning explainability to identify the most important Reynolds-stress (Q) events in a turbulent channel (simulated via DNS) and a turbulent boundary layer (obtained experimentally). This objective way to assess importance reveals that the most important Q events are not the ones with the highest Reynolds shear stress. This framework is also used to identify completely new coherent structures, and we find that the most important coherent regions in the flow only have an overlap of 70% with the classical Q events. In the second part of the presentation we use beta variational autoencoders (beta VAEs) and transformers to build reduced-order models (ROMs) for turbulent flows. The beta VAEs enable a non-linear reduced-order representation of the flow while preserving disentanglement of the latent variables. The transformer yields excellent temporal predictions in the latent space, far superior than those possible with e.g. long short-term memory (LSTM) networks, thanks to the multi-scale characteristics captured by the attention mechanisms. A combination of explainability and autoencoders with causality, to further shed light on the most important flow mechanisms, is also presented.

Seminar date and time: November 1, 2024. 10 AM ET.

Slides available here: here. Relevant videos (1,2,3,4).

Recently, advances in machine learning, hardware (e.g. GPUs/TPUs), and availability of high-quality data have set the stage for machine learning (ML) to tackle problems for weather and climate. This has led to a paradigm shift in operational weather forecasting, most evidently seen by the vast amount of resources being invested into AI models at the leading operational centers including NOAA, ECMWF, ECCC, and others.  This has been motivated by the influx of deep learning-based models in the last 3 years for weather forecasting which have been demonstrated to have forecasting skill approaching or even exceeding the best available numerical weather prediction (NWP) models. These data-driven models include Graphcast, Fourcastnet, FuXi, Pangu-weather, ClimaX, Fengwu, and Stormer, each with vastly different training methods, machine learning architectures, and variables predicted. 

In this seminar, we explore the rise of ML-based weather prediction and discuss research ongoing at Argonne that is helping revolutionize how weather and climate are researched using machine learning. Specifically, looking at: 1.) an Argonne-led project for medium-range weather forecasting called Stormer and 2.) one of the first systematic evaluations of machine learning-based emulators for climate research and several novel future research directions on creating the next generation of climate model emulators.

Seminar date and time: October 25, 2024. 10 AM ET.

Recording available here.