Call for Papers

Overview

The unprecedented scale and complexity of modern neural networks have revealed emergent patterns in learning dynamics and scaling behaviors. Recent advances in analyzing high-dimensional systems have uncovered fundamental relationships between model size, data requirements, and computational resources while highlighting the intricate nature of optimization landscapes. This understanding has led to deeper insights into the architecture design, regularization, and the principles governing neural learning at scale.

We invite participation in the 3rd Workshop on High-dimensional Learning Dynamics (HiLD) at ICML 2025. This year's theme, Navigating Complexity: Feature Learning Dynamics at Scale, explores how neural networks develop and organize representations across training of large-scale models. We encourage submissions related to our theme, as well as other topics around the theoretical and empirical understanding of learning in high-dimensional spaces. We will accept high quality submissions for poster presentations during the workshop, especially work-in-progress and state-of-art ideas. 

We welcome any topics in pursuit of understanding how model behaviors evolve or emerge. 

Example topics include but are not limited to:

• The emergence of interpretable behaviors (e.g., circuit mechanisms) and capabilities (e.g., compositionality and reasoning)

• Work that adapts tools from stochastic differential equations, high-dimensional probability, random matrix theory, and other theoretical frameworks to understand learning dynamics and phase transitions

• Scaling laws related to internal structures and functional differences

• Competition and dependencies among structures and heuristics, e.g., simplicity bias or learning staircase functions

• Relating optimizer design and loss landscape geometry to implicit regularization, inductive bias, and generalization

• Modeling of high-dimensional datasets

• Modeling of loss landscapes and simple analyzable models for deep neural networks

• Average-case analysis of optimization algorithms

• Mean field approximation regimes, neural tangent kernel, and beyond