Abstract: This talk explores the Uzawa method, tracing its development from early applications in partial differential equations (PDEs) to modern advancements in optimization, image processing, and scientific computing. We will examine recent refinements for developing GPU-adaptive solvers for huge-scale linear programming and its extension to semidefinite programming arising in quantum information science. The discussion will also highlight the method's integration with deep learning and unrolling techniques for optimal control problems of PDEs, as well as its applications in industry.

Abstract: Optimal transport (OT) has become a natural framework for guiding the probability flows. Yet, the majority of recent generative models assume trivial geometry (e.g., Euclidean) and rely on strong density-estimation assumptions, yielding trajectories that do not respect the true principles of optimality in the underlying manifold. We present Hamiltonian Optimal Transport Advection (HOTA), a Hamilton-Jacobi-Bellman based method that tackles the dual dynamical OT problem explicitly through Kantorovich potentials, enabling efficient and scalable trajectory optimization. Our approach effectively evades the need for explicit density modeling, performing even when the cost functionals are non-smooth. Empirically, HOTA outperforms all baselines in standard benchmarks, as well as in custom datasets with non-differentiable costs, both in terms of feasibility and optimality.

Abstract: Discrete diffusion models have risen as a powerful scheme for generative model on discrete data. They enable parallel, order-agnostic generation with iterative refinement, offering a flexible alternative to auto-regressive large language models (LLMs). However, controlling their output to satisfy certain conditions remains challenging. In this talk we explore two different strategies for "steering" these models towards desired properties both through training and inference.

First, we address inference-time control by formalizing how to properly extend Classifier-Free Guidance (CFG) to discrete domains, enabling robust conditioning on class labels. Second, we transition to training-time alignment under verifiable reward functions. We introduce Group Diffusion Policy Optimization (GDPO), a reinforcement learning algorithm specifically engineered for the unique transition dynamics of diffusion language models. Through a series of diverse benchmarks, we demonstrate how these methods bridge the gap between flexible diffusion generation and precise intent alignment.

Abstract: This paper investigates approximation-theoretic aspects of the in-context learning capability of the transformers in representing a family of noisy linear dynamical systems. Our first theoretical result establishes an upper bound on the approximation error of multi-layer transformers with respect to an L2-testing loss uniformly defined across tasks. This result demonstrates that transformers with logarithmic depth can achieve error bounds comparable with those of the least-squares estimator. In contrast, our second result establishes a non-diminishing lower bound on the approximation error for a class of single-layer linear transformers, which suggests a depth-separation phenomenon for transformers in the in-context learning of dynamical systems. Moreover, this second result uncovers a critical distinction in the approximation power of single-layer linear transformers when learning from IID versus non-IID data.

Abstract: Sketching methods have become a fundamental tool in the design of scalable algorithms in Randomized Numerical Linear Algebra, with broad applications in machine learning and data analysis. In this talk, I will first provide a brief introduction to sketching methods and their classical theoretical guarantees. I will then present a new perspective that formulates sketching as a functional estimation problem and introduce iterative debiasing techniques within this framework. Finally, I will describe a faster algorithm for tensor decomposition with usage of sketch method.

Abstract: Sequential Bayesian optimal experimental design (SBOED) for PDE-governed inverse problems is compu tationally challenging, especially for infinite-dimensional random field parameters. High-fidelity approaches require repeated forward and adjoint PDE solves inside nested Bayesian inversion and design loops. We formulate SBOED as a finite-horizon Markov decision process and learn an amortized design policy via policy-gradient reinforcement learning (PGRL), enabling online design selection from the experiment his tory without repeatedly solving an SBOED optimization problem. To make policy training and reward evaluation scalable, we combine dual dimension reduction—active subspace projection for the parameter and principal component analysis for the state—with an adjusted derivative-informed latent attention neu ral operator (LANO) surrogate that predicts both the parameter-to-solution map and its Jacobian. We use a Laplace-based D-optimality reward while noting that, in general, other expected-information-gain utilities such as KL divergence can also be used within the same framework. We further introduce an eigenvalue-based evaluation strategy that uses prior samples as proxies for maximum a posteriori (MAP) points, avoiding repeated MAP solves while retaining accurate information-gain estimates. Numerical ex periments on sequential multi-sensor placement for contaminant source tracking demonstrate approximately 100× speedup over high-fidelity finite element methods, improved performance over random sensor place ments, and physically interpretable policies that discover an “upstream” tracking strategy.

Abstract: The curse of dimensionality renders statistical and machine learning in high dimensions intractable without additional assumptions on the underlying data.  We consider geometric models for data that allow for mathematical performance guarantees and efficient algorithms that break the curse.  The first part of the talk develops a family of data-driven metrics that balance between density and geometry in the underlying data.  We consider discrete graph operators based on these metrics, and prove performance guarantees for clustering with them in the spectral graph paradigm.  Fast algorithms based on Euclidean nearest-neighbor graphs are proposed and connections with continuum operators on manifolds are developed. 

In the second part of the talk, we move away from Euclidean spaces and focus on representation learning of probability measures in Wasserstein space.  We introduce a general barycentric coding model in which data are represented as Wasserstein barycenters of a set of fixed reference measures.  Leveraging the geometry of Wasserstein space, we develop a tractable optimization program to learn the barycentric coordinates when given access to the densities of the underlying measures.  We provide a consistent statistical procedure for learning these coordinates when the measures are accessed only by i.i.d. samples.  Our consistency results and algorithms exploit entropic regularization of optimal transport maps, thereby allowing our barycentric modeling approach to scale efficiently.  Extensions to learning suitable reference measures and linearizations of our barycentric coding model will be discussed.  Throughout the talk, applications to synthetic and real data demonstrate the efficacy of our methods.

Abstract: Industrial Computer-Aided Engineering (CAE) has long been constrained by a structural trade-off between computational fidelity and practical usability. While high-fidelity numerical solvers remain indispensable for accuracy, their immense computational cost prohibits interactive design exploration and real-time optimization. Conversely, traditional surrogate models offer speed but lack the generalization required for diverse geometries and varying process conditions. We propose Large Engineering Models (LEMs): data-driven reference models that leverages transformer-based architectures to capture complex physical behaviors at scale. Based on the AB-UPT framework with sparse attention mechanisms, LEMs efficiently encode large-scale, unstructured 3D geometries alongside high-dimensional parameter sets.

We apply the LEM framework to plastic injection moulding, a domain characterized by tightly coupled, transient multiphysics. The model was trained on a massive dataset exceeding 100 TB of high-fidelity industrial simulations. Our results demonstrate that LEMs can accurately predict the temporal evolution of flow fronts, pressure and temperature fields. Crucially, the model achieves these results with inference times approximately three orders of magnitude faster than traditional solvers, enabling a shift to real-time generative design.

Abstract: We introduce in-context operator learning on probability measure spaces for optimal transport (OT). The goal is to learn a single solution operator that maps a pair of distributions to the OT map, using only few-shot samples from each distribution as a prompt and without gradient updates at inference. We parameterize the solution operator and develop scaling-law theory in two regimes. In the nonparametric setting, when tasks concentrate on a low-intrinsic-dimension manifold of source– target pairs, we establish generalization bounds that quantify how in-context accuracy scales with prompt size, intrinsic task dimension, and model capacity. In the parametric setting (e.g., Gaussian families), we give an explicit architecture that recovers the exact OT map in context and provide finite-sample excess-risk bounds. Our numerical experiments on synthetic transports and generative modeling benchmarks validate the framework.

Abstract: Optimal feedback control with implicit Hamiltonians poses a fundamental challenge for learning-based value function methods due to the absence of closed-form optimal control laws. Recent work by Gelphman et. al. (2025) introduced an implicit deep learning approach using Jacobian-Free Backpropagation (JFB) to address this setting, but only established sample-wise descent guarantees. In this talk, we will establish convergence guarantees for JFB in the stochastic minibatch setting, showing that the resulting updates converge to stationary points of the expected optimal control objective. We further demonstrate scalability on substantially higher-dimensional problems, including multi-agent optimal consumption and swarm-based quadrotor and bicycle control. Together, our results provide both theoretical justification and empirical evidence for using JFB in high-dimensional optimal control with implicit Hamiltonians.

Abstract: Schrödinger Bridge frameworks are widely used in generative modeling to connect stochastic distributions via the most likely path. However, existing approaches are constrained by energy-conservation assumptions, which restrict the flexibility of the bridge and limit its ability to adapt the stochastic dynamics to the available data. This hinders step-size refinement when approaching the target distribution and impairs generalization when modeling dynamics with varying energy. To address these limitations, we introduce a non-conservative Schrödinger Bridge formulation based on contact Hamiltonian mechanics. By parameterizing the probability manifold, we lift the bridge problem to a tractable geodesic computation in a finite-dimensional space, which can be naturally implemented using a ResNet architecture. The framework is validated on tasks including manifold navigation, molecular dynamics predictions, and image generation.

Abstract: Fracture presents one of the most critical challenges in engineering, directly impacting the safety, durability, and performance of structures and materials. Its complex nature makes accurate modeling a significant hurdle. Peridynamics is a powerful nonlocal reformulation of classical continuum mechanics, specifically designed to excel in simulating material failure and damage. Unlike traditional models, peridynamics naturally captures material discontinuities, such as cracks, without the need for spatial differentiability assumptions. However, the nonlocal nature of peridynamics, while highly effective, also introduces a substantial increase in computational costs compared to classical methods. To address this challenge, coupling (nonlocal) peridynamics with (local) classical continuum mechanics has emerged as an intriguing solution, combining the strengths of both approaches. Yet, local-to-nonlocal coupling brings its own set of challenges, including artifacts and the need for adaptive modeling techniques. Fortunately, recent advancements in GPU acceleration and the rise of exascale computing have made possible the development of new computational tools, providing an exciting alternative to these methods. This presentation will offer an overview of peridynamics and introduce CabanaPD, a meshfree, GPU-enabled peridynamics code designed for large-scale fracture simulations. Built on the robust libraries Kokkos and Cabana, both developed through the U.S. Department of Energy's (DOE’s) Exascale Computing Project, CabanaPD is performance-portable, exascale-capable, and optimized to run on the DOE’s supercomputers. We will demonstrate the use of CabanaPD for fracture simulation in various engineering problems. This is a cordial invitation to the following joint ACM and SIAM Student seminar.

Abstract: Presented Gaussian-mixture neural operator methods for representing complex predictive uncertainty in operator-learning problems.

Abstract: Recently, large-scale pre-trained speech encoders and Large Language Models (LLMs) have been released, which show state-of-the-art performance on a range of spoken language processing tasks including Automatic Speech Recognition (ASR). To effectively combine both models for better performance, continuous speech prompts, and ASR error correction have been adopted. However, these methods are prone to suboptimal performance or are inflexible. In this paper, we propose a new paradigm, LegoSLM, that bridges speech encoders and LLMs using the ASR posterior matrices. The speech encoder is trained to generate Connectionist Temporal Classification (CTC) posteriors over the LLM vocabulary, which are used to reconstruct pseudo-audio embeddings by computing a weighted sum of the LLM input embeddings. These embeddings are concatenated with text embeddings in the LLM input space. Using the well-performing USM and Gemma models as an example, we demonstrate that our proposed LegoSLM method yields good performance on both ASR and speech translation tasks. By connecting USM with Gemma models, we can get an average of 49% WERR over the USM-CTC baseline on 8 MLS testsets. The trained model also exhibits modularity in a range of settings -- after fine-tuning the Gemma model weights, the speech encoder can be switched and combined with the LLM in a zero-shot fashion. Additionally, we propose to control the decode-time influence of the USM and LLM using a softmax temperature, which shows effectiveness in domain adaptation.

Abstract: Sampling from constrained statistical distributions is a fundamental task in various fields, including Bayesian statistics, computational chemistry, and statistical physics. This article considers sampling from a constrained distribution that is described by an unconstrained density, as well as additional equality and/or inequality constraints, which often make the constraint set nonconvex. Existing methods struggle in the presence of such nonconvex constraints, as they rely on projections, which are computationally expensive or intractable, are specialized to either inequality or equality constraints, and often lack rigorous quantitative convergence guarantees.

In this paper, we introduce Overdamped Langevin with LAnding (OLLA), a new framework that can design overdamped Langevin dynamics accommodating both nonlinear equality and inequality constraints. The proposed dynamics also deterministically corrects trajectories along the normal direction of the constraint surface, thus obviating the need for explicit projections. We show that, under suitable regularity conditions on the target density and the feasible set $\Sigma\subset\mathbb{R}^d$, OLLA converges exponentially fast in 2-Wasserstein distance to the constrained target density $\rho_\Sigma(x) \propto \exp(-f(x))d\sigma_\Sigma$.

Lastly, through experiments, we demonstrate the efficiency of OLLA compared to known constrained Langevin algorithms and their slack variable variants, highlighting its favorable computational cost and fast empirical mixing.

Abstract: Neural operators, which map between function spaces in a discretization-independent manner, have emerged as effective tools for constructing surrogate models. However, such surrogates typically assume noise-free training data, neglecting the critical role of noise in Bayesian inverse problems. We introduce the noise aware neural operator (NANO), a novel surrogate designed to incorporate noise characteristics directly into both its architecture and training. By employing noise-weighted proper orthogonal decomposition (POD) to construct the output basis and incorporating the noise covariance into the loss function, NANO creates mappings tailored to the statistical structure of noisy data. This design improves the accuracy of posterior estimation in Bayesian inverse problems. We demonstrate NANO’s effectiveness through two examples: a sinusoidal inference problem and gravitational wave parameter estimation.

Abstract: In numerous applied research domains, including geophysics, medicine, engineering, economics, and finance, a fundamental objective is the extraction of latent information and salient features—such as quasi-periodicities and frequency patterns—as well as the decomposition of a signal into its constituent components, for instance, its underlying trend.

Classical approaches based on the Fourier and Wavelet Transforms, long established in signal processing, exhibit intrinsic limitations when addressing nonlinear and non-stationary phenomena. Consequently, over the past two decades, considerable effort has been devoted to the development of novel nonlinear methodologies by research groups worldwide, leading to their extensive application across a wide range of scientific and engineering disciplines.

In the first half of the talk, we briefly review the Fast Iterative Filtering technique and introduce a newly developed generalization, the Spherical Iterative Filtering, to handle spherical data. We will discuss the numerical convergence of this method, which relies on the Generalized Locally Toeplitz theory, and show its promising applications to real-life geophysical data.

In the second half, we introduce the Iterative Residual Convolutional Neural Network, which leverages the convolutional neural network, residual structure, and nonlinear activation function to compute, under the framework of deep learning, the non-stationary signal decomposition of a signal. We discuss the training process of the proposed model and study the convergence analysis of the learning algorithm. We present a few experiments to show the performance of the proposed model from two points of view: the calculation of the local average and the signal decomposition. Furthermore, we talk about the mode mixing, noise interference, and orthogonality properties of the decomposed components produced by the proposed method. All results show that the proposed model allows for better handling of boundary and mode mixing effects, and orthogonality of the decomposed components than existing methods, and it proves to be robust to noise perturbation.

We conclude the talk with open problems and possible future research directions.

Abstract: We will broadly introduce the concept of market making and touch upon some of the various challenges quants need to tackle to build mathematical models of the financial markets. This will be followed by brief discussion of how options work and the role and importance of volatility modeling and forecasting.

Abstract: We explore the sample-generic properties of low-complexity critical points of one hidden layer neural networks, namely, critical points representing narrower neural networks with respect to sample size. We relate the parameter and function space by identifying the sets of parameters with different complexities. Then, for the low-complexity critical sets we characterize their geometry and demonstrate the prevalence of (strict) saddles; in fact, many such sets consist solely of saddles. The analysis builds on our Dimension Killing Technique -- a technique which reveals the interplay between samples and the parameters of a parameterized model. The favorable geometry and prevalence of saddles suggest that these critical sets may not significantly impede gradient-based training.  

Abstract: To study the underlying mechanisms behind transformers and related techniques, we propose a transformer learning framework motivated by a two-stage sampling process, with distributions being inputs, and present a mathematical formulation of the attention mechanism as kernel embedding. Our findings show that by the attention operator, transformers can compress distributions into function representations without loss of information. We also demonstrate the in-context learning capabilities of efficient transformer structures through a rigorous generalization analysis.