Symmetry as Intervention; Causal Estimation with Data Augmentation (11/17/2026)
Presenter: Uzair Akbar
The technique of data augmentation (DA) is often used in machine learning for regularization purposes to better generalize under i.i.d. settings. In this work, we present a unifying framework with topics in causal inference to make a case for the use of DA beyond just the i.i.d. setting, but for generalization across interventions as well. Specifically, we argue that when the outcome generating mechanism is invariant to our choice of DA, then such augmentations can effectively be thought of as interventions on the treatment generating mechanism itself. This can potentially help to reduce bias in causal effect estimation arising from hidden confounders. In the presence of such unobserved confounding we typically make use of instrumental variables (IVs) — sources of treatment randomization that are conditionally independent of the outcome. However, IVs may not be as readily available as DA for many applications, which is the main motivation behind this work. By appropriately regularizing IV based estimators, we introduce the concept of IV-like (IVL) regression for mitigating confounding bias and improving predictive performance across interventions even when certain IV properties are relaxed. Finally, we cast parameterized DA as an IVL regression problem and show that when used in composition can simulate a worst-case application of such DA, further improving performance on causal estimation and generalization tasks beyond what simple DA may offer. This is shown both theoretically for the population case and via simulation experiments for the finite sample case using a simple linear example. We also present real data experiments to support our case.
Learning Computational Structure for Sample-Efficient Prediction (11/10/2026)
Presenter: Hangyu Zhou
Statistical learning in high dimensions suffers from the curse of dimensionality. Fortunately, many functions of practical interest are compositionally sparse, meaning they admit a composition of local functions, each depending on only a few variables, allowing sample complexity to depend on local arity rather than ambient dimension. Yet realizing this benefit requires exploiting hidden compositional structure, which theory does not establish for practical gradient-based training. We study this gap with CSFRBench, a controlled benchmark that hides the target's
compositional computation during training but retains it for diagnosis, and find that dense networks and sparsity-based baselines leave much of that benefit unrealized. Inspired by the spatial organization of biological neural circuits, we introduce Learned Wiring (LeWi): each neuron has a learned low-dimensional embedding, only nearby neurons connect, and embeddings and
weights are trained jointly. LeWi improves sample efficiency on targets with hierarchical structure, intermediate reuse, or independent modules. Across 32 additional targets generated by independently sampled computation graphs, LeWi is the most sample-efficient method not given the target graph, with 27% lower mean learning-curve area than dense networks. Dense networks perform better on targets with a single high-arity function. LeWi's representations and connectivity align with the target computation. On 17 world-model prediction tasks spanning control, navigation, and manipulation, LeWi beats the dense baseline on 16 and wins 99/119 baseline comparisons. These results show that jointly learning connectivity and weights can exploit computational structure to improve sample efficiency.
Learning Not to Interfere: Adapting Experts for Model Merging (11/03/2026)
Presenter: Pratik Ramesh
Model merging combines experts fine-tuned from a shared pretrained initialization into a single multitask model, but overlapping parameter updates often degrade their specialized capabilities. We argue that this interference stems from how experts are fine-tuned. Standard fine-tuning optimizes each expert on its own task distribution without constraining how its parameter update affects predictions on other tasks. These parameter updates, known as task-vectors, represent changes from the shared initialization and can disrupt other experts when combined. We propose Resolving Interference (RI), a lightweight framework that independently adapts experts before merging using unlabeled task data. RI distills the original expert’s predictions on its own task to preserve specialization, while matching the shared pretrained model’s predictions on other tasks. The pretrained model provides a reference for behavior before specialization, allowing RI to restrict each task vector’s influence beyond its own task distribution. RI complements existing merging methods, including those using test-time adaptation, and improves performance across vision and language benchmarks. For language-model merging, RI surpasses the strongest evaluated baselines by up to 2.9%. Additionally, quantitative consistency evaluations and qualitative generations confirm RI’s ability to better preserve the original experts’ behavior across tasks.
Chance-Constrained Safe Reinforcement Learning under Unknown Disturbances via Conformal Prediction (10/27/2026)
Presenter: Stratos Reppas
We propose a stochastic optimal control framework based on Conformal Prediction (CP) that relaxes common restrictive assumptions such as known noise distributions or zero-mean Gaussian noise, using finite system data to quantify uncertainty in a distribution-free manner. Under an additive i.i.d. disturbance hypothesis, we derive a closed-form optimal control policy for discrete-time LTI systems that enforces probabilistic safety under arbitrary, potentially non-zero-mean disturbances.
We then move beyond this hypothesis to a reinforcement learning (RL) setting in which only an imperfect model of the nonlinear system is available, so that disturbance and model errors together make the calibration data state-, input-, and policy-dependent. This exposes a tension inherent to combining CP with policy learning: the i.i.d. assumption drops, so residuals along a trajectory are temporally correlated rather than exchangeable, and policy-induced distribution shifts mean that every improvement step invalidates the quantile computed before it, leaving subsequent policies possibly uncertified. We resolve this with Trajectory-Informed Safe Policy Iteration, which uses data gathered along multiple trajectories, restoring exchangeability, and integrates a CP-based penalty into the policy improvement step, so that the quantile and the policy co-converge and the final controller is provably certified. The result is a principled route to use imperfect world models to learn policies with probabilistic safety guarantees.
Flash Talk Seminar (10/13/2026)
Presenters: Adrian Ho, Nathan Derhake, Ziyu Zhu, Haoran Yan
First-Order Bilevel Optimization for Non-Truthful Auction Design (Adrian Ho)
Auditability and the Landscape of Distance to Multicalibration (Nathan Derhake)
New Results from Testing Frontier Agents (Ziyu Zhu)
Neural Scaling Laws of Transformer Operator Network (Haoran Yan)
Teaching Graph Neural Networks to Think Globally (09/22/2026)
Presenter: Nyambura Njenga
The development of self-supervised graph pre-training methods is a crucial ingredient in recent efforts to design robust graph foundation models (GFMs). Structure-based pre-training methods are under-explored yet crucial for downstream applications which rely on underlying graph structure. In addition, pre-training traditional message passing GNNs to capture global and regional structure is often challenging due to the risk of oversmoothing as network depth increases. We address these gaps by proposing the Laplacian Eigenvector Learning Module (LELM), a novel pre-training module for graph neural networks (GNNs) based on predicting the low-frequency eigenvectors of the graph Laplacian. Moreover, LELM introduces a novel architecture that overcomes oversmoothing, allowing the GNN model to learn long-range interdependencies. Empirically, we show that models pre-trained via our framework outperform baseline models on downstream molecular property prediction tasks.
This presentation is based on our recent work presented at TAG-DS 2025.