Call for Papers
Important Dates
Notification Timeline
Deadline for submission of papers: May 27, 2024, anywhere on earth (*)
Notification of acceptance: June 17, 2024
Camera-Ready Papers: July 19, 2024 (High-dimensional learning dynamics style file required) (**)
Workshop date July 26, 2024
(*) If you face severe difficulties meeting this deadline, please contact us before the deadline.
Submission of papers will be through OpenReview and limited to no more than 5 pages plus supplementary materials.
We are not an archival proceedings. Check with the other journals/conferences, but this usually means you can submit to us and the journal/conference without violating dual submissions. For example, you may submit to us and NeurIPS without violating its dual submission policy.
All submissions must be anonymized and may not contain any identifying information that may violate the double-blind reviewing policy.
For accepted workshop posters, please adhere to the following:
Dimensions 36 in (H) x 24 in (W) or 91 cm (H) x 61 cm (W); this differs from the Main Conference
Portrait format
Information as well as printing services available by ICML can be found here: https://icml.cc/Conferences/2024/PosterInstructions
Invited Talks
Talks will be in-person and live-streamed
Angelica Chen (NYU)
Aukosh Jagannath (U of Waterloo)
Jason Lee (Princeton)
Kanaka Rajan (Harvard)
Pragya Sur (Harvard)
Lenka Zdeborová (EPFL)
Overview
Modeling learning dynamics has long been a goal of the empirical science and theory communities in deep learning. These communities have grown rapidly in recent years, as our newly expanded understanding of the latent structures and capabilities of large models permits researchers to study these phenomena through the lens of the training process. Recent progress in understanding fully trained models can therefore enable understanding of their development and lead to insights that improve optimizer and architecture design, provide model interpretations, inform evaluation, and generally enhance the science of neural networks and their priors. We aim to foster discussion, discovery, and dissemination of state-of-the-art research in high-dimensional learning dynamics relevant to ML.
We invite participation in the 2nd Workshop on High-dimensional Learning Dynamics (HiLD), to be held as a part of the ICML 2024 conference. This year’s theme focuses on understanding how reasoning capabilities and internal structures develop over the course of neural network training; we encourage submissions related to our theme as well as other topics around the theoretical and empirical understanding of learning in high dimensional spaces. We will accept high quality submissions as poster presentations during the workshop, especially work-in-progress and state-of-art ideas.
We welcome any topics in pursuit of understanding how model behaviors evolve or emerge.
Example topics include but are not limited to:
The emergence of interpretable behaviors (e.g., circuit mechanisms) and capabilities (e.g., compositionality and reasoning)
Work that adapts tools from stochastic differential equations, high-dimensional probability, random matrix theory, and other theoretical frameworks to understand learning dynamics and phase transitions
Scaling laws related to internal structures and functional differences
Competition and dependencies among structures and heuristics, e.g., simplicity bias or learning staircase functions
Relating optimizer design and loss landscape geometry to implicit regularization, inductive bias, and generalization
Modeling of high-dimensional datasets
Modeling of loss landscapes and simple analyzable models for deep neural networks
Average-case analysis of optimization algorithms
Mean field approximation regimes, neural tangent kernel, and beyond