UCSB Applied Math/PDE/Data Science Seminar

Abstract: One of the most popular approaches for TV-regularized optimization problems in the space of measures is the so-called Particle Gradient Flow. For this, one restricts to linear combinations of Dirac deltas and then takes a Euclidean gradient flow in the weights and positions, significantly simplifying computations. Recent results have shown that PGFs recover Wasserstein gradient flow dynamics, and can even converge to global minima under sufficient conditions. In this talk, I present a generalization of PGFs to regularized optimization problems on arbitrary Banach spaces, which we call Atomic Gradient Flow (AGF). The crucial idea is the choice of the right notion of particles, which we argue to be the extremal points of the unit ball of the regularizer. Using Choquet's theorem, we lift the problem into the Wasserstein space of both weights and extremal points, and study convexity, existence, and uniqueness properties for the AGF and metric gradient flows in the lifted setting. Our main result is that the lifting of the AGF is again a metric gradient flow in the Wasserstein space, implying that the AGF follows a very strong dynamic. Lastly, I will also showcase examples, applications and some numerics. This is joint work with Marcello Carioni and Konstantinos Zemas.

Host: Katy Craig

Abstract: Many emerging uses of AI, from analyzing scientific literature to developing complex software, require models to work with large amounts of information. Yet the ability to accept a long input does not explain how effectively a model uses it. What mathematical principles determine the capabilities and limitations of large language models as their context grows? In this talk, I will explore this question through mathematical models of transformers, the architecture underlying many language models, with self-attention at its core. I will first study context-length-dependent temperature scaling, a practical technique used in serveral language models to improve retrieval from long inputs. Drawing on extreme-value theory, we provide a mathematical explanation for this largely empirical strategy by identifying the critical scaling that allows multiple strong candidates to influence the output, even as the context grows. I will then turn to in-context learning, the remarkable ability of an LLM to learn from examples in inputs without changing its parameters. A longer context can contain more examples and therefore more information, making it important to understand how rapidly a model can benefit from this growing body of evidence. Using mean-field analysis, we quantify how quickly learning from a finite context approaches learning from the underlying true population. I will conclude by placing these results within my broader research on mathematical machine learning, highlighting opportunities for applied mathematics to guide the design of reliable learning algorithms.

Host: Xu Yang

Abstract: Hack's Law says that the length of the main river in a river basin scales with the area of the river basin to the power 0.58, thereby prompting speculation on whether there exist optimality principles governing natural processes like channel formation.  Identification of such principles and their continued mathematical development could lead to the formulation of a theoretical framework which supports study not only of fluvial landscape evolution, but other geomorphological processes as well.  Erosion is nature's process of ``moving dirt." In 1781, Monge initiated the theory of optimal transport with the question, ``Given a pile of sand and a pit of equal volume, how can one optimally transport the sand into the pit?"  Building on earlier work by Birnir, Merchant, Smith, Cattan, and Rowlett, we aim to extend optimal transport theory to develop mathematical models and computational methods for studying erosion. Via optimal transport theory, we prove the existence of unique global weak solutions to equations describing the sediment flow in the evolution of fluvial land surfaces, with constant water depth.   While an earlier existence theory result by Birnir and Landry demonstrates that the slopes, or gradients of the surface, are moved optimally, our current work proves that erosion does what Monge proposed: move sediment optimally from the mountainside to the river.  This is joint work with Bjorn Birnir. 

Abstract: Large language models (LLMs) can use examples provided in a prompt to make predictions for new queries without updating their parameters, a capability known as in-context learning. In this talk, I will explore how transformers, the architectural backbone of many LLMs, can perform nonlinear kernel ridge regression (KRR). I will first present numerical observations showing that transformers pretrained on regression tasks match KRR predictions increasingly well as depth grows. Motivated by these observations, I will present a theoretical construction showing how a transformer can approximate the Gaussian KRR predictor through a connection with a convergent iterative method for solving the underlying kernel system. In this construction, successive transformer layers approximate iterations of the solver, progressively refining the KRR solution. Together, these results provide theoretical and empirical insights into in-context learning with transformers for nonlinear regression.

Abstract: In this talk we will discuss the role of the forward and inverse energy cascades for non-uniqueness, blow-up, or global existence of smooth solutions of the unforced 3D Navier-Stokes equations. For smooth initial data, inverse energy cascade can result in an instantaneous blow-up and produce smooth for positive time solutions that enjoy the same regularity properties as the Koch-Tataru solutions. For distributional initial data, forward energy cascade can lead to enhanced dissipation and produce global in time very weak solutions. We characterize the class of distributional initial data for which classical weak solutions exist. In particular, any divergence-free initial datum in W^{s,p}, gives rise to a global weak solution, smooth for positive time, if and only if s>=-1. For instance, take any divergence-free initial datum that belongs to W^{-1,p} for some finite p>1 but not H^{-1}. Then the solution to the heat equation is not in L^2_{x,t}. On the other hand, the constructed weak solution is in L^2_{x,t}, as well as C([0,infinity);W^{-1,p}), and smooth for positive time. Hence, the NSE solutions can enjoy better integrability than the solutions of the heat equation. The nonlinear term regularizes the solution via a forward energy cascade, which also results in enhanced dissipation and hence better integrability than the heat flow.

Host: Ruimeng Hu

Abstract: The shapes of cellular membranes emerge from forces generated by the organization of lipids and proteins, yet connecting these molecular-scale forces to organelle-scale morphology remains a central challenge. I will describe our efforts to bridge these scales using forward physics-based models along with new strategies for extracting ultrastructure from 3D electron microscopy data in the hopes of inferring material properties from shape. First, I will present the development of agent-based actin simulations coupled with a deformable membrane model to explain how interfilament sterics provides the bracing necessary for the directional motility of giant unilamellar vesicles. Next, turning to interactions between organelles, I will ask whether the geometries of large organelles such as the yeast nucleus and vacuole can be explained by steric crowding alone. Finally, I will introduce a method for building physics-guided implicit neural representations of membranes from 3D electron tomograms, producing smooth, simulation-ready geometries that link imaging directly to mechanics.

Host: Paul Atzberger