This workshop is  motivated by recent advances in the rapidly developing area of foundation models - i.e., large-scale neural network models pretrained in an unsupervised way on very large and diverse datasets. Such models often demonstrate significant improvement in their few-shot generalization abilities, as compared to their smaller-scale counterparts, across a wide range of downstream tasks. One of the main themes  of this particular workshop is on emergent behaviours and phase transitions in deep learning.  

Scaling and emergence are of great importance to both AI capabilities and AI Alignment and Safety. We plan to discuss recent developments along both direction.

A set of questions we aim to address in the workshop includes, but is not limeted to, the following: 

1. What are the underlying mechanisms and principles that govern the behavior of machine learning models?

2. When and under what conditions can scaling laws be applied to machine learning models?

3. What are the critical behaviors of machine learning models near phase transitions, and how can these be-

haviors be explained and predicted?

4. Can we find progress measures that underlie sudden performance improvements? See Nanda et al. (2023), Barak et al. (2022).

5. What are the future directions and opportunities for research in emergent phenomena in machine learning, and how can this research contribute to the development of more robust and generalizable machine learning models?