This is the 8th workshop in the Scaling workshop series that started in Oct 2021. It provides a forum for discussing the challenges and advances in scaling foundation models. This workshop, co-organized by Cerebras Systems, MBZUAI and the CERC in Autonomous AI Lab led by Irina Rish at the Universite de Montreal and Mila - Quebec AI Institute will focus on both development and deployment aspects of large-scale models. 

While methodologies for pretraining large-scale models continue to evolve with ongoing improvements in data efficiency, architecture design, and optimization, post-training and reinforcement learning are now becoming central to enhancing model capabilities and alignment. At the same time, production inference has become increasingly important for bringing large models into production, facing memory bandwidth limitations, latency requirements for real-time applications, and cost constraints. 

This workshop focuses on bridging the gap between academic research and production reality by bringing together machine learning researchers, model builders, industry practitioners, hardware innovators, and systems researchers. We will explore efficient deployment strategies, novel architectures, and co-design approaches that make large models both practical and performant in production environments.