Call for Papers

Overview

The unprecedented scale and complexity of modern neural networks have revealed emergent patterns in learning dynamics and scaling behaviors. Recent advances in analyzing high-dimensional systems have uncovered fundamental relationships between model size, data requirements, and computational resources while highlighting the intricate nature of optimization landscapes. This understanding has led to deeper insights into the architecture design, regularization, and the principles governing learning at scale.

We invite participation in the 4th Workshop on High-dimensional Learning Dynamics (HiLD) at ICML 2026. This year's theme, scaling laws, explores universalities and the scaling of large-scale machine learning models. We encourage submissions related to our theme, as well as other topics around the theoretical and empirical understanding of learning in high-dimensional spaces. We will accept high quality submissions for poster presentations during the workshop, especially work-in-progress and state-of-art ideas. 

We welcome any topics that help us understand how model behaviors evolve at a large scale. 

Example topics include, but are not limited to:

• Compute-optimal scaling laws: theory and practice

• Effect of optimizers, hyperparameters, and batch size on scaling exponents

• Feature learning and representation in scaling regimes

• Neural scaling laws from statistical mechanics and random matrix theory

• Data-constrained and multi-epoch scaling

• Architecture-dependent scaling (transformers, state-space models, mixture-of-experts)

• High-dimensional limits of stochastic optimization algorithms

• Mean-field and continuous-time limits of learning dynamics

• Loss landscape geometry at scale (Hessian spectra, edge of stability)