Part 1:  Overlaps with the 9am-5pm workshop at Mila (Agora) on

Reality+Virtual Worlds and the Problems of Philosophy   (here are recording of the workshop)

____________________________________________________________________________________________________________

Part 2.  After the workshop,   the class continues from 5:30 to 6:30pm                                          (Auditorium 2 at  Mila; streaming link).

Lecturer: Irina Rish
Topic: Brief intro & overview  of the class (video)

Lecturer: Irina Rish 

Topic: Neural Scaling Laws: History and Overview (slides, video )

Papers:  The Bitter Lesson,    GPT-3 paper: Language Models are Few-Shot Learners,                   Scaling  Laws for Neural Language Models,     Training compute-optimal large language     

models (summary/blog: New Scaling Laws for LLMs)

Lecturer: Irina Rish 

Topic: Phase Transitions in AI (and Emergent Behaviors in Large-Scale models) (slides, video)

The Law of Robustness by Sebastien Bubeck

Papers:  Hard and Easy Distributions of SAT Problems,  Every Monotone Graph Property Has a Sharp Threshold,  Approximability of probability distributions,   Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets,   GPT-3 paper: Language Models are Few-Shot Learners,   A universal law of robustness via isoperimetry.

Part 1:    Lecturers:  Kyle Roth, Alex Fulleringer

Topic: Unsolved Problems in ML Safety (slides, video - part 1)

____________________________________________________________________________________________________________________

Part 2:   Lecturers: Mohammad Samsami, Sonia Joseph

Topic: A Generalist Agent (Gato)  (slides, video - part  2)

Part 1:    Lecturers:  Hamza Abdelhedi, Jordan O'Byrne

Topic:  Optimal neural representation in neural systems at the edge of chaos  (slides, video - part 1)
Background:  Reservoir computing, Echo state network,  Next generation reservoir computing,  Echo State Neural Machine Translation

Fri Feb 10:  conversation on emergence, intelligence and transhumanism with Prof. Michael Levin and Machine Learning Street Talk

____________________________________________________________________________________________________________

Part 2:    Lecturers:  Maryam Valipour, Farzad Salajegheh

Topic:  data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language (slides, video - part 2)

 Part 1:  Lecturers:  Vitaly Kondulukov, Kaushik Moudgalya

Topic:  Perceiver: General Perception with Iterative Attention  (slides, video - part 1)

____________________________________________________________________________________________________________

Part 2:  Lecturers:  Alireza Razagh, Leo Gagnon

Topic: MusicLM: Generating Music From Text (slides, video - part 2)

 Part 1:  Lecturers:  Artem Zholus 

Topic:  Emerging Properties in Self-Supervised Vision Transformers   (slides, video -part 1)

____________________________________________________________________________________________________________

Part 2:  Lecturers:  Charles-Étienne Joseph, Pulkit 

Topic: Chain of Thought Prompting Elicits Reasoning in Large Language Models  (slides, video - part 2)

 Part 1:  Lecturers:  Maryam Valipour, Farzad Salajegheh 

Topic:  A ConvNet for the 2020s    (slides, video -part 1)

____________________________________________________________________________________________________________

Part 2:  Lecturers:  Rohan Banerjee, Selim Gilon 

Topic: Masked Autoencoders Are Scalable Vision Learners  (slides, video - part 2)

 Part 1:  Lecturers:  Alireza Razaghi, Nizar Islah 

Topic:  Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts  (slides, video -part 1)

____________________________________________________________________________________________________________

Part 2:  Lecturers:  Mohammad Samsami, Sonia Joseph 

Topic: Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small   (slides, video - part 2)

Part 1:  Lecturers:  Rohan Banerjee, Selim Gilon

Topic:  Effect of Pre-Training Scale on Intra- and Inter-Domain Full and Few-Shot Transfer Learning for Natural and Medical X-Ray Chest Images   (slides, video -part 1)

____________________________________________________________________________________________________________

Part 2:  Lecturers:  XinyiHE, Johan S

Topic: Constitutional AI: Harmlessness from AI Feedback    (slides, video - part 2)

Part 1:  Lecturers:  Chuanrui Wang, Huiyu Cai

Topic:  Evolutionary-scale prediction of atomic-level protein structure with a language model (slides, video -part 1)

____________________________________________________________________________________________________________

Part 2:  Lecturers:  Sparsha Mishra 

Topic: Artificial muses: Generative Artificial Intelligence Chatbots Have Risen to Human-Level Creativity     (slides, video - part 2)

Part 1:  Lecturer:  Léo Gagnon

Topic: Data Distributional Properties Drive Emergent In-Context Learning in Transformers

 (slides, video -part 1)

_______________________________________________________________________________________________________

Part 2:  Lecturer:  Artem Zholus

Topic: Visual Prompt Tuning     (slides, video - part 2)

Part 1:  Lecturers:  Johan S and XinyiHE

Topic: Collaborating with language models for embodied reasoning (slides, video -part 1)

_______________________________________________________________________________________________________

Part 2:  Lecturers:  Arkil Patel and Tejas Vaidhya

Topic: LoRA: Low-Rank Adaptation of Large Language Models  (slides, video - part 2)

Part 1:  Lecturer:  Sangnie Bhardwaj and Sarthak Mittal

Topic: BayesFlow: Learning complex stochastic models with invertible neural networks   (slides, video )

Part 1:  Lecturer:  Leo Gagnon

Title : In-context learning of causal adaptation strategies

Slides: here

 Video : here

_______________________________________________________________________________________________________

Part 2  Lecturer: Farzad Salajegheh, Maryam Valipour, Alireza Razzaghi 

Title : Deformable gate in CNNs

Slides: here

 Video : here

_____________________________________________________________________________________________________

Part 3  Lecturer: Niki Howe

Title : Scaling scaling laws with board games

Slides: here

 Video : here

_______________________________________________________________________________________________________

Part 4 Lecturer: Huiyu Cai, Jiarui Lu, Zuobai Zhang

Title : Exploring scaling behavior for protein language models

Slides: here

 Video : here

Part 1:  Lecturer:  Johan Samir Obando Ceron

Title : Unlocking the Neural Capacity for Sample Efficient Reinforcement Learning

Slides: here

 Video : here

_______________________________________________________________________________________________________

Part 2  Lecturer: Kyle Roth, Georges Bélanger, Alex Fulleringer

Title : "Fine-Tuning" a frozen LLM by teaching it to record information to external tools

Slides: here

 Video : here

_______________________________________________________________________________________________________

Part 3  Lecturer: Alexandre Larouche, Ali Touil

Title : Scaling laws for Coorperation in Multi-Agent RL 

Slides: here

 Video : here

_______________________________________________________________________________________________________

Part 4  Lecturer: Vamsikrishna Chemudupati, Sandeep Kumar

Title : Investigating Slimmable networks for transformer-based models in ASR

Slides: here

 Video : here

_______________________________________________________________________________________________________

Part 5 Lecturer: Gwen LeGate, Charles-Étienne Joseph, Albert Orozco Camacho

Title : Scaling Federated Learning

Slides: here

 Video : here

_______________________________________________________________________________________________________

Part 6 Lecturer: Hamza Abdelhedi, Jordan O'Byrne

Title : Brain similarity metrics 

Slides: here

 Video : here

Part 1:  Lecturer:  Alekhya Dronavalli, Praneet Suresh 

Title : Impact of Scale in Multi-task Variational Information Bottleneck

Slides: here

 Video : here

_______________________________________________________________________________________________________

Part 2  Lecturer: Tejas Vaidhya, Arkil Patel 

Title : Pruned LLMs

Slides: here

 Video : here

Part 1:  Lecturer:  Alekhya Dronavalli, Praneet Suresh 

Title : Contrastive Syn-to-Real Generalization

Slides: here

 Video : here