September 9

Ben Brindle

• Title: Clustering Irregularly Sampled Time Series Data

• Abstract: We explore how to create a framework to algorithmically cluster irregularly sampled multivariate time series using latent variable models. Motivated by real-world examples, we describe different model formulations. For a continuous latent variable, we use linear Gaussian state space models. In this case, we outline our algorithm and explain how it uses the Kalman filter and Rauch-Tung-Striebel smoother. For a discrete latent variable, we draw from continuous-time hidden Markov models. We introduce a novel approach to model and cluster multivariate time series data in this setting and discuss its compatibility with existing literature. For both cases, we provide an overview of the underlying theory, most importantly the expectation maximization algorithm.

September 16

Hadden Kim

• Title: Solving differential equations with non-linear models

• Abstract: I'll describe a general method to numerically solve evolutional, differential equations using non-linearly parameterized models. In particular, we can use evolutional deep neural networks (EDNN) to solve PDE problems, such as Navier Stokes. In the last three years, several interpretations of this method were published, connecting us to broader research communities. This method can be viewed as a finite element method, as traversing on an ansatz manifold, as a shape-morphing mixture model, as an active machine learning process, and as a natural gradient descent algorithm.

September 23

Thabo Samakhoana

• Title: The Optimal Smoothing of Sublinear Functions and Convex Cones

• Abstract: We consider the following natural question: for a given nonsmooth convex function of interest, how closely can we approximate such a function using smooth, convex functions of bounded curvature? In this talk, we will present exact characterization of solutions to this problem when the nonsmooth function is sublinear. We do the same for convex cones.  We will then show how our theory provides insights into smoothing more general convex functions and sets. Finally, we will present how these smoothings can be used to accelerate some first order optimization methods.

September 30

Debsurya De

• Title: Computational Lower Bounds for Low-Rank Estimation Under Inhomogeneous Noise. 

• Abstract: Recent work has generalized several results concerning the classical spiked Wigner matrix model of a rank-one signal matrix corrupted by additive i.i.d. Gaussian noise to the inhomogeneous case, where the noise has a variance profile. In particular, for the special case where the variance profile has a block structure, a series of results identified an effective spectral algorithm for detecting or recovering the signal, identified the threshold signal strength required for this algorithm to succeed, and shown that, for some distributions of a random signal, this algorithm is information-theoretically optimal. We complement these results by studying the computational optimality of this spectral algorithm, in particular showing that low-degree polynomial algorithms cannot improve its performance for detection for a wide range of signal distributions. We also show that these lower bounds are universal with respect to the observation channel, for instance applying equally well to the degree-corrected stochastic block model and provide some new information-theoretic lower bounds sharpening ones previously known. Notably, unlike almost all prior work on this model, our results do not assume that the variance profile has a block structure and suggest that the same spectral algorithm might in fact be optimal for a much broader range of inhomogeneous noise models.

October 7

Parik Chalise

• Title: A Combinatorial Interpretation of Matrices

• Abstract: What is a matrix? The answer depends on who you ask. We provide one more: A matrix is a union of rainbow subgraphs of a complete bipartite graph. Then we redefine matrix multiplication and interlacing inequalities with this perspective.

Aaron Zoll

• Title: Using ChatGPT To Solve Interesting Jane Street Puzzles

• Abstract: The title is all you get. 

October 14

Aaron Zoll

• Title: A Universal Optimal Gradient Method

• Abstract: The minimization of Hölder smooth convex functions provides a family of problems interpolating from smooth to nonsmooth Lipschitz. This has proven to be a fruitful model for the design of universal, adaptive algorithms. In this talk, we extend recent advances in computer-assisted design for first-order optimization over structured functions–known as Performance Estimation Problems (PEP)–to apply to Hölder smooth functions. More generally, we extend the results witnessed in the novel ""Optimized Gradient Method"" to a universal setting for the class of inexactly smooth functions.  This new theory is based on characterizations of ``approximate cocoercivity'' for associated tolerance parameters ε. As an application of this new PEP machinery, we identify new best-known methods for Hölder smooth minimization (requiring careful selection of both stepsizes and tolerances ε), improving on prior worst-case guarantees by a constant factor.

October 21

Michelle Dai

• Title: Option Pricing Using Feedforward Neural Networks (i.e. What is financial mathematics?)

• Abstract:  This talk aims to be an introduction to the field of financial mathematics and how tools such as neural networks are applied in this context. Predicting the future performance of time-varying processes is crucial to many fields of research, especially in finance, where we focus on the pricing of stock options. Early option models relied on past price and volatility values to calculate prices directly, but they evolved to focus more on modeling implied volatility, the metric traders use most in practice. Parametric models incur error and are inflexible to different types of options; nonparametric models have high computational demands and are difficult to interpret. Our compromise is adopting a parametrically-guided nonparametric approach: we use the ad-hoc Black-Scholes (AHBS) model to fit an implied volatility surface, then use feedforward neural networks to further reduce parametric modeling error. Applying this model on a diversity of stocks represented in the S&P100, we see that nonparametric correction efforts vary based on the nature of their respective sectors; most surprisingly, stocks expected to be stable during financial crises actually proved more difficult to model. As for the performance of our model, we find that specifying an appropriate underlying parametric baseline, as well as finding the ideal dimensions of our neural network, are doubly important for accurate price prediction for various types of options.

October 28

Adam Tsou

• Title: What are Compound AI Systems? 

• Abstract: A compound AI system is defined as a system with multiple interacting components such as LLMs, retrieval modules, and multi-step chains. While these models have been able to achieve performance beyond that of traditional LLMs, optimizing them is a challenge due to their inherently more complex nature. This talk will primarily focus on introducing compound AI systems, and their components such as RAG (retrieval augmented generation), and multi-step chains. We will also begin to introduce some tools for optimization in compound AI systems such as LLMSelector, which helps to select the optimal LLM for each task in a compound system.

November 4

Mary Versa Clemens-Sewall

• Title: Random Partition Based Models Under Distribution Shift

• Abstract: Using machine learning models in the real world requires training them on available data and trusting their performance on real inputs they'll receive wherever they're used. This talk takes one of the problems that can occur--distribution shifts--and returns it to the cozy realm of theory for us to learn from it. A distribution shift occurs whenever the training data are not identically distributed to a test point. In this talk, we'll analyze how distribution shifts affect performance guarantees for the purely random forest model, a variant of the popular random forest model.

November 11

Alan Luner

• Title: An Adaptive Subgame Perfect Gradient Method

• Abstract: We introduce a new algorithm for smooth convex optimization, the Adaptive Subgame Perfect Gradient Method (ASPGM), which combines recent advances in subgame perfection, linesearch-free adaptivity, and auto-conditioning. At each iteration, ASPGM makes a momentum-type update, optimized dynamically based on a (limited) memory/bundle of past first-order information. In addition, ASPGM is parameter-free and linesearch-free. In this talk, I will gradually construct our method and explain the different ideas that contribute to the method.

November 18

Wilson Gregory

• Title: Any Dimensional PDE Surrogates

• Abstract: Many PDEs can be written as an equation of fields, Laplacians, gradients, curl, divergence, and higher order terms of these differential operators. These operators are defined for any dimension, which allows the same equation to be written for any dimension. For machine learning surrogate models however, we must learn in a particular dimension in order to apply our model in that dimension. If we could learn using these any dimensional operators, then we could learn in a low dimension (say 1 or 2) and transfer our model to a higher dimension (such as 2 or 3). Our prior work on O(d)-equivariant tensor image models seems to have learned discrete versions of Laplacians and gradients, giving us a possible path forward.

December 2

Beatrix Wen

• Title: Galois Theory Challenges Weisfeiler Leman: Invariant Features for Symmetric Matrices and Point Clouds

• Abstract: Recent work [Blum-Smith et al., 2024] proposed an invariant machine learning model on point clouds and symmetric matrices based on invariant features. For symmetric matrices, the model is invariant under the group action of permutation by conjugation. Therefore, this model can be used to test whether two graphs are isomorphic (by checking whether the invariant features of two graphs coincide). In this work, we investigate the expressive power of this new method. Our theoretical results show that on undirected graphs, the method is strictly worse than graph neural networks (GNNs) based on message passing. To improve its expressivity, we propose a modified method by adding a new invariant function, which we test empirically against GNNs and other baseline methods. The newly proposed function set outperformed the original invariant features, yielding compatible or exceeding results compared to GNNs. Joint work with Caio Netto and Thabo Samakhoana.