AI & Scale Workshop Schedule
Day 1: Thu, March 28th
morning session: Agora: 9:30am - 1:00pm
9:45 - 10:45 An Intro: On Scaling Laws, Foundation Models and HPC Irina Rish (CERC-AAI Lab Lead, UdeM/Mila) slides / video
An overview of Large-Scale AI Projects at CERC-AAI Lab and AGI Collective
break/coffee/Q&A
11:00 - 12:00 A tutorial on HPC & Building Foundation Models Quentin Anthony (CERC-AAI/OSU/Eleuther/Zyphra) slides / video
For more background, you can also see tutorials at our course webpage HPC, Large-Scale Models
12:00 - 1:00 Cerebras Systems: Hardware, Software, Scaling Laws and Sparsity (Natalia Vassilieva, Joel Hestness) slides / video (starting at: 01:01:53)
lunch
afternoon session: A14, Mila 6666: 2:00pm - 5:00pm
2:00 -5:00 A tutorial on HPC & Building Foundation Models: Part 2 Quentin Anthony slides / video 1, video 2, video 3
__________________
Day 2: Fri, March 29th (Agora)
In-depth Overview of Large-Scale AI Projects at CERC-AAI Lab and AGI Collective; Discussion of Open Questions, Next Steps, and Collaborations
10:00 - 10:30 Time-Series Foundation Models Arjun Ashok, Roland Riachi slides / video (paper, blog, tweet)
10:30- 11:00 Continual Pretraining of Foundation Models Benjamin Thérien, Adam Ibrahim, Kshitij Gupta slides / video (paper, blog, tweet)
break/coffee/Q&A
11:10 - 11:40 Robin Suite of Open-Source Multimodal Foundation Models Kshitij Gupta, Daniel Kaplan slides / video (paper, blog, tweet)
11:40 - 12:10 Multimodal Alignment: Towards ethical multimodal systems Alexis Roger slides / video (starting at 00:18:58 ) (paper1, paper2)
12:10 - 12:30 The New Frontier of Generated Data Diganta Misra slides / video (paper, tweet)
lunch (pizza)
1:00 - 1:30 LLM 4 Psychology and Psychology 4 LLMs Tommaso Tosato
1:30 - 2:00 Compression and Fast Inference in Foundation Models Tejas Vaidhya slides / video (paper , blog, tweet)
break/coffee/Q&A
2:00 - 3:00 LLMs through the HPC Lens Quentin Anthony (CERC-AAAI Lab/OSU/Eleuther/Zyphra) slides / video
3:00 - 5:00 Q&A with Quentin: in person and online
Relevant materilas from previoius scaling workshop:
an overview of large-scale AI projects at CERC-AAI Lab and AGI Collective presented in Dec at the 6th Neural Scaling Workshop in New Orleans, co-located with NeurIPS 2023
Irina Rish: Open-Source Foundation Models on Supercomputers: projects and models built by CERC-AAI Lab and INCITE 2023 Collab
Quentin Anthony (EleutherAI & CERC AAI Lab) EleutherAI: DL Research In the Open (video, slides)
Kshitij Gupta, Benjamin Thérien, Adam Ibrahim et al: Continual Pretraining of Foundation Models (15min) (paper, blog, tweet, slides, video)
Kshitij Gupta, Daniel Kaplan: Robin Suite of Open-Source Multimodal Foundation Models (15 min) (paper, blog, tweet, slides, video)
Alexis Roger: Multimodal Alignment: Towards ethical multimodal systems (10 min) (paper, blog, tweet, slides, video)
Arjun Ashok, Andrew Williams: Time-Series Foundation Models (15 min) (paper , blog, tweet, slides, video)
Nolano.ai (Tejas Vaidhya, Ayush Kaushal, Irina Rish) (video)
Ayush Kaushal Nolano: Compression and Fast Inference in Foundation Models (8 min) (paper , blog, tweet, slides)
Ayush Kaushal: Nolano: Introducing Hi-NOLIN - the First Hindi-English LLM (5 min) ( tweet, blog, slides)