The Schedule

A PAC-Bayesian Perspective on the Interpolating Information Criterion

Graph Neural Networks Benefit from Structural Information Provably: A Feature Learning Perspective

Linear attention is (maybe) all you need (to understand transformer optimization) 

Large Catapults in Momentum Gradient Descent with Warmup: An Empirical Study 

Feature Learning in Infinite-Depth Neural Networks

Variational Classification

Implicit biases in multitask and continual learningfrom a backward error analysis perspective

Spectrum Extraction and Clipping for Implicitly Linear Layers

The Noise Geometry of Stochastic Gradient Descent: A Quantitative and Analytical Characterization

Curvature-Dimension Tradeoff for Generalization in Hyperbolic Space

Complexity Matters: Dynamics of Feature Learning in the Presence of Spurious Correlations 

Unveiling the Hessian's Connection to the Decision Boundary 

Nonparametric Classification on Low Dimensional Manifolds using Overparameterized Convolutional Residual Networks

Large Learning Rates Improve Generalization: But How Large Are We Talking About?

Understanding the Role of Noisy Statistics in the Regularization Effect of Batch Normalization 

Generalization Guarantees of Deep ResNets in the Mean-Field Regime

Theoretical Explanation for Generalization from Adversarial Perturbations

In-Context Convergence of Transformers

How Two-Layer Neural Networks Learn, One (Giant) Step at a Time

Two Facets of SDE Under an Information-Theoretic Lens: Generalization of SGD via Training Trajectories and via Terminal States

Unraveling the Complexities of Simplicity Bias: Mitigating and Amplifying Factors

Transformers as Support Vector Machines

Symmetric Mean-field Langevin Dynamics for Distributional Minimax Problems

A Theoretical Study of Dataset Distillation

Transformers Learn Higher-Order Optimization Methods for In-Context Learning: A Study with Linear Models

Introducing an Improved Information-Theoretic Measure of Predictive Uncertainty

In-Context Learning on Unstructured Data: Softmax Attention as a Mixture of Experts

Attention-Only Transformers and Implementing MLPs with Attention Heads

Privacy at Interpolation: Precise Analysis for Random and NTK Features

Denoising Low-Rank Data Under Distribution Shift: Double Descent and Data Augmentation

A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networks

Benign Overfitting and Grokking in ReLU Networks for XOR Cluster Data 

How does Gradient Descent Learn Features --- A Local Analysis for Regularized Two-Layer Neural Networks

Understanding Transferable Representation Learning and Zero-shot Transfer in CLIP

Provably Efficient CVaR RL in Low-rank MDPs 

Analysis of Task Transferability in Large Pre-trained Classifiers

On Scale-Invariant Sharpness Measures

Gibbs-Based Information Criteria and the Over-Parameterized Regime

Grokking modular arithmetic can be explained by margin maximization

Over-parameterised Shallow Neural Networks with Asymmetrical Node Scaling: Global Convergence Guarantees and Feature Learning

On the Computational Complexity of Inverting Generative Models 

Flow-Based High-Dimensionally Distributional Robust Optimization

Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining

How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations 

A Theoretical Explanation of Deep RL Performance in Stochastic Environments 

Deep Networks as Denoising Algorithms: Sample-Efficient Learning of Diffusion Models in High-Dimensional Graphical Models 

Under-Parameterized Double Descent for Ridge Regularized Least Squares Denoising of Data on a Line

Continual Learning for Long-Tailed Recognition: Bridging the Gap in Theory and Practice

SimVAE: Narrowing the gap between Discriminative & Generative Representation Learning 

Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks 

Benign Oscillation of Stochastic Gradient Descent with Large Learning Rate

On Compositionality and Emergence in Physical Systems Generativie Modeling

Escaping Random Teacher Initialization Enhances Signal Propagation and Representations 

The Expressive Power of Transformers with Chain of Thought

Transformers as Multi-Task Feature Selectors: Generalization Analysis of In-Context Learning

Fit Like You Sample: Sample-Efficient Score Matching From Fast Mixing Diffusions

Towards the Fundamental Limits of Knowledge Transfer over Finite Domains

Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization 

MoXCo:How I learned to stop exploring and love my local minima?

First-order ANIL provably learns representations despite overparametrisation

A Data-Driven Measure of Relative Uncertainty for Misclassification Detection

Non-Vacuous Generalization Bounds for Large Language Models

Learning from setbacks: the impact of adversarial initialization on generalization performance

Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit 

Estimating optimal PAC-Bayes bounds with Hamiltonian Monte Carlo 

Divergence at the Interpolation Threshold: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle

Good regularity creates large learning rate implicit biases: edge of stability, balancing, and catapult

Toward Student-oriented Teacher Network Training for Knowledge Distillation

Adaptive Sharpness-Aware Pruning for Robust Sparse Networks 

Invariant Low-Dimensional Subspaces in Gradient Descent for Learning Deep Matrix Factorizations

How Structured Data Guides Feature Learning: A Case Study of the Parity Problem

The Next Symbol Prediction Problem: PAC-learning and its relation to Language Mode

Why Do We Need Weight Decay for Overparameterized Deep Networks?

The Double-Edged Sword: Perception and Uncertainty in Inverse Problems ls 

Near-Interpolators: Fast Norm Growth and Tempered Near-Overfitting

On robust overfitting: adversarial training induced distribution matters

Are Graph Neural Networks Optimal Approximation Algorithms? 

JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention