The unprecedented scale and complexity of modern machine learning models have revealed emergent patterns in learning dynamics and scaling behaviors. Recent advances in analyzing high-dimensional systems have uncovered fundamental relationships between model size, data requirements, and computational resources while highlighting the intricate nature of optimization landscapes. This understanding has led to deeper insights into the architecture design, regularization, and the principles governing neural learning at scale.  The HiLD workshop seeks to spur research and collaboration around the following: 

1. Compute-optimal scaling laws: theory and practice

2. Effect of optimizers, hyperparameters, and batch size on scaling exponents

3. Feature learning and representation in scaling regimes

4. Neural scaling laws from statistical mechanics and random matrix theory

5. Data-constrained and multi-epoch scaling

6. Architecture-dependent scaling (transformers, state-space models, mixture-of-experts)

7. High-dimensional limits of stochastic optimization algorithms

8. Mean-field and continuous-time limits of learning dynamics

9. Loss landscape geometry at scale (Hessian spectra, edge of stability)

This year, the 4th Workshop on High-dimensional Learning Dynamics theme is on Scaling laws. We aim to foster discussion, discovery, and dissemination of state-of-the-art research on modern machine learning models, scaling, and high-dimensional learning dynamics.  Bringing together experts from all parts of ML: theorists and empirical scientists.  It seeks to create synergies between these two groups, which often do not interact. Through a series of talks, the workshop will tackle questions on high-dimensionality and scaling in ML.