Workshop Schedule

Day 1: Monday 27 June 2022

• 8:45-9:00 meeting at Mila

• 9:00-10:00 Bus to Manoir St Sauveur

• 10:00-10:30 Coffee

• 10:45-11:00 Welcome (Rachade)

• 11:00-11:45 Introduction and Overview Irina Rish (Mila/ UdeM) video/slides

• 11:45-12:30 workshop participants: introductions

• 12:30 -13:30 Lunch

• 13:30-14:00 A discussion on AI Safety and Alignment David Krueger (U of Cambridge)

• 14:00-14:15 Coffee Break

• 14:15-15:30 The Unreasonable Effectiveness of Mathematics in Large-Scale Deep Learning Greg Yang (Microsoft) video/slides

• 16:00-18:00 Check-in/Free Time

• 18:30-20:00 Talks (Students: 3 talks/20min each):

  • The Primacy Bias in Deep Reinforcement Learning Evgenii Nikishin (Mila) video/slides

  • The Hidden Scaling Capabilities of Deep Reinforcement Learning Algorithms Pierluca D'oro (Mila)

  • Revisiting Learnable Affines for Batch Norm in Few-Shot Transfer Learning Muawiz Chaudhary (MIla)

• 19:30-21:00 Dinner

Day 2: Tuesday 28 June 2022

• 07:30-09:00 Breakfast

• 9:15-10:00 Towards Understanding Grokking: An Effective Theory of Representation Learning Eric Michaud (MIT) video/slides

• 10:00-10:15 Coffee Break

• 10:15-11:15 Extracting learned knowledge from a black box Max Tegmark (MIT) video

• 11:15-11:45 Discussion on Emergent Behaviors & Phase Transitions in Large Scale Models (Eric, Max, Guillaume, Irina)

• 11:45-12:00 Coffee Break

• 12:00-13:00 Sources of Variance in Pretraining and Finetuning Naomi Saphra (NUY)

• 13:00:-14:00 Lunch

• 14:00-17:00 Free Time

• 17:00-18:00 Engineering Challenges and Techniques involved in Training Multi-Billion Parameter Transformers Sandeep Subramanian (NVIDIA) video/slides

• 18:00-18:30 Coffee Break

• 18:30-19:00 Anytime Progressive Pruning Diganta Misra (Mila) video/slides

• 19:00-20:30 Dinner

Day 3: Wednesday 29 June 2022

• 07:30-09:00 Breakfast

• 9:20-9:40 Continual Learning for Language Models Evangelia Gogoulou video/slides

• 9:40-10:30 Capturing AGI’s Value: The Case for Applications Brady Neal (Mila/Oogway) video/slides

• 10:30 Coffee

• 10:40-11:30 Updates on Collaboration (Mila, LAION, Eleuther, stability.ai) video/slides (LAION, Stability.ai)

• 11:30-12:00 check out

• 12:00-13:00 Lunch

• 13:00 - 13:30 David Dohan (Google) video/slides

• 13:30 - 13:45 The AGI Tech Stack Sonia Joseph video/slides

• 13:45 - 14:15 Summary & Future Steps/ Wrap up discussion

• 14:15-17:00 free time

• 17:00 bus back to Montreal (if you prefer to stay longer, please make your own arrangements/carpool plans)