Continual Learning: Towards “Broad” AI
IFT 6760B Winter 2021, Université de Montréal / Mila - Quebec AI Institute
Lecture 2 (18/01/2021)
Part 1: Lecturer: Irina Rish
Slides: Meta-Learning and Continual Learning
Video: here (part 1)
Part 2: Lecturer: Matthew Riemer
Slides: Meta-Experience Replay
Paper: Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference (ICLR 2019)
Video: here (part 2)
Lecture 3 (21/01/2021)
Part 1: Invariant Risk Minimization (IRM) and Invariant Risk Minimization Games (IRM Games)
Lecturer: Kartik Ahuja
Slides: here
Video: here
Part 2: Kartik's talk, continued. Followed by brief remarks (Irina) on ILC and other recent ood methods
Lecturer: Kartik Ahuja and Irina Rish
Slides: here
Lecture 5 (28/01/2021)
Part 1: IIRC: Incremental Implicitly-Refined Classification
Lecturers: Sarath Chandar (Assistant Professor at École Polytechnique de Montréal).
Video: here (part 1)
Part 2: Towards Continual RL
Lecturers: Matt Riemer
Slides: here
Video: here (part 2)
Lecture 6 (01/02/2021)
Part 1: Variational Continual Learning (VCL)
Lecturer: David Yu-Tung Hui and Max Schwarzer
Slides: here
Video: here (part 1)
Part 2: iCaRL: Incremental Classifier and Representation Learning
Lecturers: Reza Davari and Mathieu Béligon
Slides: here
Video: here (part 2)
Lecture 7 (04/02/2021)
Part 1: Learning Without Forgetting (LwF)
Lecturer: Irene Tenison and Sai Aravind Sreeramadas
Slides: here
Video: here (part 1)
Part 2: Sequoia - The Research Tree (https://github.com/lebrice/Sequoia)
Lecturer: Fabrice Normandin
Slides: here
Video: here (part 2)
Lecture 8 (08/02/2021)
Part 1: A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning
Lecturer: Amin Mansouri and Nadir Hassen
Slides: here
Video: here (part 1)
Part 2: Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting
Lecturer: Arnold (Zicong) Mo and Yusong Wu
Slides: here
Video: here (part 2)
Lecture 9 (11/02/2021)
Part 1: PackNet (Adding Multiple Tasks to a Single Network by Iterative Pruning)
Lecturer: Mostafa Elaraby and Dishank Bansal
Slides: here
Video: here (part 1)
Part 2: Ternary Feature Masks: continual learning without any forgetting
Lecturer: Marie St-Laurent and Shima Rastegarnia
Slides: here
Video: here (part 2)
Lecture 10 (15/02/2021)
Part 1: Learning without Memorizing
Lecturer: Nader Asadi and Arian Khorasani
Slides: here
Video: here (part 1)
Part 2: Continual Learning Through Synaptic Intelligence
Lecturer: Jean-Christophe Gagnon-Audet
Slides: here
Video: here (part 2)
Lecture 11 (18/02/2021)
Part 1: Alleviating catastrophic forgetting using contextdependent gating and synaptic stabilization
Lecturer: Corentin Moiny and Maryam Ghaderi
Slides: here
Video: here (part 1)
Part 2: Meta-Learning Symmetries by Reparameterization
Lecturer: Pierre-André Brousseau and Guillaume Lam
Slides: here
Video: here (part 2)
Lecture 12 (22/02/2021)
Part 1: On Tiny Episodic Memories in Continual Learning
Lecturer: Darshan Patil and Ali Rahimi-Kalahroudi
Slides: here
Video: here (part 1)
Part 2: Memory Aware Synapses: Learning what (not) to forget
Lecturer: Olivier Tessier-Larivière and Nikky Runghen-Vézina
Slides: here
Video: here (part 2)
Lecture 13 (25/02/2021)
Part 1: Functional regularisation for continual learning with gaussian processes
Lecturer: Nadhir Hassen and Raghav Gupta
Slides: here
Video: here (part 1)
Part 2: Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Lecturer: Dishank Bansal and Mostafa Elaraby
Slides: here
Video: here (part 2)
Lecture 14 (11/03/2021)
Part 1: Never-Ending Learning
Lecturer: Brady Neal and Joshua Jacobs
Slides: here
Video: here (part 1)
Part 2: End-to-End Incremental Learning
Lecturer: Reza Davari and Mathieu Béligon
Slides: here
Video: here (part 2)
Lecture 15 (15/03/2021)
Part 1: Learning Independent Causal Mechanisms
Lecturer: Michelle Liu and Paul Crouther
Slides: here
Video: here (part 1)
Part 2: Calibrating CNNs for Lifelong Learning
Lecturer: Pierre-André Brousseau and Guillaume Lam
Slides: here
Video: here (part 2)
Lecture 16 (18/03/2021)
Part 1: Optimizing for the Future in Non-Stationary MDPs
Lecturer: Max Schwarzer and David Yu-Tung Hui
Slides: here
Video: here (part 1)
Part 2: BiC (Large Scale Incremental Learning)
Lecturer: Ali Rahimi-Kalahroudi and Raghav Gupta
Slides: here
Video: here (part 2)
Lecture 17 (22/03/2021)
Part 1: Brain-inspired replay for continual learning with artificial neural networks
Lecturer: Marie St-Laurent and Shima Rastegarnia
Slides: here
Video: here (part 1)
Part 2: Learning Causal Models Online
Lecturer: Sean Spinney and Michelle Liu
Slides: here
Video: here (part 2)
Lecture 18 (25/03/2021)
Part 1: Continual Reinforcement Learning with Complex Synapses
Lecturer: Darshan Patil and Rupali Bhati
Slides: here
Video: here (part 1)
Part 2: Dark Experience for General Continual Learning: a Strong, Simple Baseline
Lecturer: Olivier Tessier-Larivière and Nikky Runghen-Vézina
Slides: here
Video: here (part 2)
Lecture 19 (25/03/2021)
Part 1: Expert Gate: Lifelong Learning with a Network of Experts
Lecturer: Remus Mocanu and Paul Crouther
Slides: here
Video: here (part 1)
Part 2: CRIB (Incremental Object Learning from Contiguous Views)
Lecturer: Maryam Ghaderi and Irene Tenison
Slides: here
Video: here (part 2)
Lecture 19 (25/03/2021)
Part 1: Risk Extrapolation
Lecturer: Ethan Caballero and Joshua Jacobs
Slides: here
Video: here (part 1)
Part 2: Using Hindsight to Anchor Past Knowledge in Continual Learning
Lecturer: Nader Asadi and Arian Khorasani
Slides: here
Video: here (part 2)
Lecture 19 (08/04/2021)
Scaling Laws for Neural Language Models
Lecturer: Brady Neal and Ethan Caballero
Slides: here
Video: here
Lecture 20 (12/04/2021)
Part 1: Domain-Free Adversarial Splitting for Domain Generalization
Lecturer: Sai Aravind Sreeramadas
Slides: here
Video: here (part 1)
Part 2: Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks
Lecturer: Remus Mocanu and Rupali Bhati
Slides: here
Video: here (part 2)
Lecture 21 (15/04/2021)
Part 1: Differentiable Causal Discovery from Interventional Data
Lecturer: Amin Mansouri and Sean Spinney
Slides: here
Video: here (part 1)
Part 2: Summary and Conclusion
Lecturer: Irina Rish
Slides: here
