Class 1 (Mon, Jan 8, 4:30-6:30pm)

Lecturer: Irina Rish 

Topic: The Large-scale AI Revolution: Breakthroughs and Challenges    (slides)

Papers:  The Bitter Lesson,   GPT-3 paper: Language Models are Few-Shot Learners

Class 3 (Mon, Jan 15, 4:30-6:30pm)

Part 1: Lecturer: Irina Rish 

Topic: Introduction to Continual Learning (slides)

Papers:  Continual T0,   Effect of scale on catastrophic forgetting in neural networks  (summary: Effects of Model and Prior Learning Scale on Catastrophic Forgetting),   Foundational Models for Continual Learning: An Empirical Study of Latent Replay

Part 2: Lecturer: Irina Rish 

Topic: Continual Learning at Scale
Papers:  Continual Pre-Training of Large Language Models: How to (re)warm your model.  

Blog: Continual Learning of Foundation Models 

Class 5  (Mon, Jan 22,  4:30-6:30pm)

Lecturer: Irina Rish 

Topic: Phase Transitions in AI &Emergent Behaviors in Large-Scale models (slides, video)

Tutorial on Phase Transitions (by Guillaume Dumas, at  the 2nd Workshop on Neural Scaling Laws)

Talk by Pascal Jr. Tikeng Notsawo (University of Montreal/Mila): Is grokking predictable?  video Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grok 

The Law of Robustness by Sebastien Bubeck

Papers:  Hard and Easy Distributions of SAT Problems,  Every Monotone Graph Property Has a Sharp Threshold,  Approximability of probability distributions,   Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets,   GPT-3 paper: Language Models are Few-Shot Learners,   A universal law of robustness via isoperimetry

Class 6  (Mon, Jan 29, 4:30-6:30pm)

Part 1: Lecturers:  Benjamin Therien and Avery Ryoo

Topic :  Scaling  Laws for Neural Language Models  (slides, video)

Part 2: Training Compute-Optimal Large Language Models  (Chinchilla Explained: video)

Class 8  (Mon, Feb 5, 4:30-6:30pm)

Part 1: Lecturers:  Paria Mehrbod and Vaibhav Singh

Topic:  Scaling Laws for Transfer   (slides, video)  Passcode: +U^C8Zfp

Part 2: Lecturer: Irina Rish

Topic:  Computational Psychoilogy and Foundation Models intersection, and related class projects (video)

Papers:  Huang et al (2023) Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBench
Gupta et al (2023) Investigating the Applicability of Self-Assessment Tests for Personality Measurement of Large Language Models.  

Explore PsychoBench

AI alignment: A comprehensive survey - TBA

Class 10  (Mon, Feb 12,  4:45-6:30pm)

Part 1: Lecturers:    Tanner Ducharme and Mahmood Hegazy 

Topic:   Chain of Thought Prompting Elicits Reasoning in Large Language Models (slides, video)   Passcode: wpPHJ1H#

Part 2: Lecturer: Moetez Kdayem 

Topic:  Vision Mamba : Efficient Visual Representation Learning with Bidirectional State Space Model
VMamba: Visual State Space Model   (slides, video-part1, video-part2)

Class 12  (Mon, Feb 19,  4:30-6:30pm)

Part 1: Lecturers:   Istabrak Abbes, Megh Thakkar

Topic: Wide Neural Networks Forget Less Catastrophically     (slides, video)    

Part 2: Lecturer: Paloma Fernandez, Anthony Gosselin  

Topic:   Unsolved Problems in ML Safety  (slides, video) 

Class 14  (Mon, Feb  26  4:30-6:30pm)

Part 1: Lecturers:    William Callaghan  and Humza Wajid Hameed 

Topic:    Artificial Intelligence, Values, and Alignment   (slides, video - 1st hr)    

Part 2:  Lecturers: Moksh Jain and Vedant Shah  (slides, video - 2nd hr)    

Topic:   Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback 

Class 14  (Wed, March 6,  4:30-6:30pm)

Part 1: Lecturers:  Shruti Joshi  and  Aniket Didolkar  

Topic:  Mamba: Linear-Time Sequence Modeling with Selective State Spaces (slides, video - part 1)

Part 2:  Lecturers:   Anirudh  and Dhuruva Priyan 

Topic:   Optimal neural representation in neural systems at the edge of chaos  (slides, video - part 2)

Class 15  (Mon, March 11,  4:30-6:30pm)

Part 1: Lecturers:  Misha Barth  and  Sammy Sharief 

Topic:  GENERALIZATION IN DIFFUSION MODELS ARISES FROM GEOMETRY-ADAPTIVE HARMONIC REPRESENTATION  ON MEMORIZATION IN DIFFUSION MODELS (slides, video)

Part 2:  Lecturers:   Yuchen Hui and Yicong Li  (slides, video)

Topic:   Mixtral of Experts 

Class 16  (Wed, March 13,  4:30-6:30pm)

Part 1: Lecturers:  Paolo Cudrano

Topic:  Effect Of Model And Pretraining Scale On Catastrophic Forgetting In Neural Networks (slides, video)

Part 2:  Lecturers:   Mahsan Abdoli, Prince Immanuel

Topic:   Constitutional AI: Harmlessness from AI Feedback (slides, video) 

Class 17  (Mon, March 18,  4:30-6:30pm)

Part 1: Lecturers: Nicolas Bernier

Topic: Scaling Data-Constrained Language Models (slides, video)

Class 18  (Wed, March 20,  4:30-6:30pm)

Part 1: Lecturers: Jon Pilarte Bilbao, Karoline Lippert

Topic:  The power of quantum neural networks (slides, video)

Part 2:  Lecturers:    Aloys Portafaix, Congshu Zou

Topic:   UniT: Multimodal Multitask Learning with a Unified Transformer (slides, video)

Class 19  (Mon, March 25,  4:30-6:30pm)

Part 1: Lecturers: Aniket Saxena, Venkatesh Ramesh

Topic:  Retrieval-Augmented Multimodal Language Modeling (slides, video)

Part 2:  Lecturers:   Jeremy Qin, Jinghan Sun

Topic:   In deep reinforcement learning, a pruned network is a good network (slides, video - part 2)

Class 20  (Wed, March 27,  4:30-6:30pm)

Part 1: Lecturers: Darsh Kaushik, Xavier Morin

Topic:  LoRA: Low-Rank Adaptation of Large Language Models (slides, video)

Part 2:  Lecturers:    Jerome Francis, Raj Ghughare

Topic:   Beyond neural scaling laws: beating power law scaling via data pruning (slides, video)

Class 21  (Mon, Apr 1,  4:30-6:30pm)

Part 1: Lecturers: 

Topic: 

Part 2:  Lecturers:  

Topic:   

Class 22  (Wed, Apr 3,  4:30-6:30pm)

Part 1: Lecturers: Meng Cao

Topic:  The Quantization Model of Neural Scaling (slides, video)

Class 23  (Mon, Apr 8,  4:30-6:30pm)

Part 1: Lecturers: Rishika Bhagwatkar, Shubham Gupta

Topic:  Perceiver: General Perception with Iterative Attention (slides, video)

Class 24  (Wed, Apr 10,  4:30-6:30pm)

Part 1: Lecturers: Andrei Mircea

Topic:  Maximal update parameterization / transfer (slides, video)

Part 2:  Lecturers:    Adel Nabli, Johan Obando

Topic:   DiLoCo: Distributed Low-Communication Training of Language Models (slides, video)

Class 25  (Mon, Apr 15,  4:30-6:30pm)

Part 1: Lecturers: Maxime Gevers

Topic:  Segment Anything (slides, video)

Part 2:  Lecturers:    Nithya Shikarpur, Subhrajyoti Dasgupta

Topic:   Adding Conditional Control to Text-to-Image Diffusion Models (slides, video)

Class 26  (Wed, Apr 17,  4:30-6:30pm)

Part 1: Lecturers: Juan David Vargas

Topic:  Recurrent Independent Mechanisms (slides, video)

Class 27  (Mon, Apr 22,  4:30-6:30pm)

Part 1: Lecturers: Sarvjeet Singh Ghotra

Topic:  Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos (slides, video)

Part 2:  Lecturers:    Sparsha Mishra, Maxime Gevers

Topic:   When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment (slides, video)

Class 28  (Wed, Apr 24,  4:30-6:30pm)

Part 1: Lecturers: Neeraj Kumar, Paul Janson

Topic:  Würstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models (slides, video)

Part 2:  Lecturers:    Ayoub Echchahed, Nassim El Massaudi

Topic:   Stop Regressing: Training Value Functions via Classification for Scalable Deep RL (slides, video)