AI @ Scale

CERC-AAI Event: Workshop, Tutorials and Q&A

March 28-29 @ Mila Registration Schedule Call-in link Photos

Scaling workshop series are organized by the CERC in Autonomous AI Lab led by Irina Rish at the Universite de Montreal and Mila - Quebec AI Institute.

The goal of these events is to provide a forum for discussing recent advances in foundation models - large-scale neural networks, pretrained in an unsupervised way on large and diverse datasets - and neural scaling laws for predicting models' behaviors at scale.

This event aims to provide:

An overview of the rapidly growing field of large-scale AI, foundation models, scaling laws and emergent behaviors
Tutorials on how to build such models in multi-GPU settings (see some background here on HPC and Distributed Model Training)
An overview of ongoing open-source foundation models projects at CERC-AAI lab and AGI Collective discord community, based on continual pretraining, multimodal model training an alignment, as well as time-series research. Thanks to the OLCF compute on Summit and Frontier supercomputers, so far, these efforts resulted into several open-source model releases, including:
1. six LLMs of 7B to 10B parameters (including 7B Red-Pajama-INCITE released May 2023, a CL-FoMo suite of fours 9.6B LLMs released in December 2023, as well as Hi-NOLIN - the first English/Hindi LLM, released in OCtober 2023);
2. Robin suite of varying-size vision-language models (VLMs) released in December 2023; ongoing efforts on multimodal evaluation and alignment
3. Time-series foundation model (Lag-Llama) released in February 2024, and more!

Page updated

Report abuse