Modeling learning dynamics has long been a goal of the empirical science and theory communities in deep learning. These communities have grown rapidly in recent years, as our newly expanded understanding of the latent structures and capabilities of large models permits researchers to study these phenomena through the lens of the training process. Recent progress in understanding fully trained models can therefore enable understanding of their development and lead to insights that improve optimizer and architecture design, provide model interpretations, inform evaluation, and generally enhance the science of neural networks and their priors.  The HiLD workshop seeks to spur research and collaboration around: 

1. Developing analyzable models and dynamics to explain observed deep neural network phenomena;

2. Competition and dependencies among structures and heuristics, e.g., simplicity bias or learning staircase functions; 

3. Creating mathematical frameworks for scaling limits of neural network dynamics as width and depth grow;

4. Provably explaining the role of the optimization algorithm, hyper-parameter choices, and neural network architectural choices on training/test dynamics;

5. Relating optimizer design and loss landscape geometry to implicit regularization, inductive bias, and generalization;

6. High-dimensionality, where intuitions from low-dimensional geometry tend to lead to inaccurate (and often misleading) properties of the machine learning models on large real-world datasets;

7. Connecting model architectures and data distributions to generalization, memorization, and forgetting.

This year the 2nd Workshop on High-dimensional Learning Dynamics theme is on The Emergence of Structure and Reasoning. We aim to foster discussion, discovery, and dissemination of state-of-the-art research in high-dimensional learning dynamics relevant to ML and bring together experts from all parts of ML: theorists to the empirical scientists.  It seeks to create synergies between these two groups which often do not interact. Through a series of talks, the workshop will tackle questions on high-dimensionality in ML.