The unprecedented scale and complexity of modern neural networks have revealed emergent patterns in learning dynamics and scaling behaviors. Recent advances in analyzing high-dimensional systems have uncovered fundamental relationships between model size, data requirements, and computational resources while highlighting the intricate nature of optimization landscapes. This understanding has led to deeper insights into the architecture design, regularization, and the principles governing neural learning at scale.  The HiLD workshop seeks to spur research and collaboration around: 

1. Developing analyzable models and dynamics to explain observed deep neural network phenomena;

2. Competition and dependencies among structures and heuristics, e.g., simplicity bias or learning staircase functions; 

3. Creating mathematical frameworks for scaling limits of neural network dynamics as width and depth grow;

4. Provably explaining the role of the optimization algorithm, hyper-parameter choices, and neural network architectural choices on training/test dynamics;

5. Relating optimizer design and loss landscape geometry to implicit regularization, inductive bias, and generalization;

6. High-dimensionality, where intuitions from low-dimensional geometry tend to lead to inaccurate (and often misleading) properties of the machine learning models on large real-world datasets;

7. Connecting model architectures and data distributions to generalization, memorization, and forgetting.

This year, the 3rd Workshop on High-dimensional Learning Dynamics theme is on Navigating Complexity: Feature Learning Dynamics at Scale. We aim to foster discussion, discovery, and dissemination of state-of-the-art research in high-dimensional learning dynamics relevant to ML and bring together experts from all parts of ML: theorists to empirical scientists.  It seeks to create synergies between these two groups, which often do not interact. Through a series of talks, the workshop will tackle questions on high-dimensionality in ML.