Neural Scaling & Alignment

Towards Maximally Beneficial AGI  

 CERC AAI Lab Workshop Series

Upcoming events (details TBA) :
July 26th, 2024 - 7th Scaling Workshop, co-located with ICML 2024, Vienna, Austria

This week:
Scaling Projects  Workshop

Overview and Goals

This workshop series are motivated by recent advances  in the rapidly developing area of foundation models -  large-scale neural networks such as GPT-3, 4 etc - pretrained in an unsupervised way on very large and diverse datasets. Such models often demonstrate significant improvement in their few-shot generalization abilities, as compared to their smaller-scale counterparts, across a wide range of downstream tasks - what one could call a "transformation of quantity into quality" or an "emergent behavior". This is an important step towards a long-standing objective of achieving Artificial General Intelligence (AGI). By AGI here we mean literally a "general", i.e. broad, versatile AI capable of quickly adapting to a wide range of situations and tasks, both novel and those encountered before - i.e. achieving a good stability (memory) vs plasticity (adaptation) trade-off, using the continual learning terminology. 

The goal of the workshop series is to provide a forum for discussion of the  most recent advances in large-scale pretrained models, focusing specifically on empirical scaling laws of such systems' performance, with increasing compute, model size, and pretraining data (power laws, phase transitions). We will also explore the trade-off between the increasing AI capabilities and AI safety/alignment with human values, considering a range of evaluation metrics beyond the predictive performance. These topics are also closely  related to  transfer-, continual- and meta-learning, as well as out-of-distribution generalization, robustness and invariant/causal predictive modeling.

Previous  Workshops

AI @ Scale   ( Mila-Agora and onlineThu Mar 28 - Fri Mar 29, 2024)

CERC AAI: INCITE Offsite (Mt Tremblant, AX Hotel, Feb 10-12, 2024)

6th Neural Scaling Workshop (New Orleans, LA, USA, Dec 15, 2023) @NeurIPS 2023 

CERC AAI  ComPsy-FoMo: Computational Psychology and Foundation Models  ( 6650 St Urbaine, Montreal, Nov 30, 2023)

CERC AAI Scientific Retreat on  Scaling & Alignment  (Mont-Tremblant, Le Grand Lodge, QC, Canada, Nov 3-5,  2023)

CERC AAI:  Offsite Lab Meeting (Mt Tremblant, Le Westin, Oct 21-22, 2023)

5th Neural Scaling Laws workshop (Honolulu, USA, July 28, 2023) @ ICML 2023

4th Neural Scaling Laws workshop (New Orleans,  LA, USA, Dec 2nd 2022) @NeurIPS 2022

3rd Neural Scaling Laws workshop (Saint-Sauveur, QC, Canada, June 27-29, 2022)

2nd Neural Scaling Laws workshop (Mont-Tremblant, QC, Canada, Dec 14-17, 2021)

1st Neural Scaling Laws workshop  (Mont-Tremblant, QC, Canada, Oct 20-22, 2021)

Related Events at the CERC AAI Lab  

Other AI Scaling & Alignment Workshops