Community - Summer School

At Cohere Labs Open Science Community, we believe the future of machine learning isn’t gated by credentials, borders, or budgets, it’s built in open collaboration, powered by curiosity, and made stronger through community.

This summer, we’re excited to launch the Cohere Labs Open Science Community Summer School, a learning initiative featuring some of the leading minds in machine learning from INRIA, META (FAIR), Google DeepMind, Cohere Labs and more.

Participants will receive a digital certificate recognizing their attendance and participation in the Cohere Labs Open Science Community Summer School.

These speakers have shaped the field across areas like foundation models, retrieval, multimodal learning, and AI for social impact. Now, they’re coming together to share their insights with you! This summer school is a chance to take the first (or next) step, alongside a global community of learners and builders.

This initiative reflects the core mission of Cohere Labs: supporting fundamental research, expanding access, and enabling the next generation of ML thinkers — regardless of where they start. We are very grateful to our community leads for organizing this series of events.

FAQ's ❓

How can I join the Cohere Labs Discord?

Please fill out the application here - note that these applications are reviewed manually and the acceptance to the Discord server is not automatic. You will receive an email with a discord link.

Will the sessions be recorded?

Yes! All sessions will be recorded and uploaded on this webpage below within 2-3 business days of the session taking place. You can also find all recordings on our Youtube playlist here.

How will attendance be tracked?

We will receive an attendance report from Google Meet automatically after each session.

Join us on Discord

The best way to stay connected during the ML Summer School is through the Cohere Labs open science community Discord server!

Once you're in:

Navigate to #ml-summer-school to meet other participants, ask questions, and get real-time updates.
Be sure to add the “ml-summer-school” role to your profile (top left menu “Channels & Roles”). This way, you will get notified about all Summer School events and activities.

Schedule of Speakers

July 2nd

Katrina Lawrence,
Applied Mathematician

ML Math Refresher

This math refresher will leave you confident to take on machine learning problems! We will build your mathematical foundation and cover the following concepts: derivatives, vector calculus, and linear algebra. Learning these concepts will allow you to understand the math upon which many algorithms are built. To get a head start, be sure to check out my math YouTube channel: https://www.youtube.com/@MathUnlockedWithKatrina

Session Recording

Q&A answered from session

July 2nd

Nils Reimers,
VP AI Search @ Cohere

Introduction to Embeddings & Retrieval

Learn about Embeddings and Retrieval from Nils reimers, VP of AI Search at cohere.com. His team develops state-of-the-art methods for search using (large) transformer networks using various techniques (dense & sparse embeddings, neural re-ranking, query generation). The most common challenge of applying neural search models is the lack of labeled training data in most domains. Hence, they develop novel strategies how to bridge this gap by combining the strengths of various approaches.

Session Recording

July 3rd

Siddhant Gupta,
NLP Community Lead, Cohere Labs

Introduction to Transformers and evolution of large language models

We will dive into the foundational principles of transformer architecture — covering attention mechanisms, positional encoding, and model scaling, tracing their breakthroughs in evolution in deep learning. Whether you’re new to LLMs/NLP or looking to deepen your understanding, this session will equip you with a solid grasp of how modern LLMs are built.

Session Recording

July 6th

Timothée Darcet,
PhD @ Meta AI (FAIR) x Inria

Scaling Self-Supervised Learning for Vision: An Introduction to DINOv2

In this session, we will explore the fundamentals of self-supervised learning (SSL) in computer vision, a powerful approach that enables models to learn useful image representations without labeled data. We’ll start with the core principles and motivations behind SSL, introduce the main families of methods such as contrastive learning and masked image modeling, and progressively build toward understanding the architecture and training strategies of DINOv2, a state-of-the-art SSL model for visual representation learning. By the end, you'll have a clear picture of how DINOv2 works and how it's used in real-world applications like image retrieval, classification, or segmentation.

Session Recording

July 8th

Lawrence Jang,
Researcher at META

A Temperature Check on Web Agents

With the rise of large language models, we've entered an era where AI systems can not only understand language—but also use it to take action. One especially exciting frontier is web agents: LLM-powered systems that can autonomously browse, click, scroll, and even fill out forms. This talk offers a fast-paced overview of where web agents stand today, what they've learned to do (and what they haven’t), and the key challenges ahead as we teach models to operate effectively and responsibly on the open web.

Session Recording

July 10th

Isha Puri,
AI PhD @MIT

Test-Time Scaling Small LMs to o1 level

As LLMs hit diminishing returns from traditional growth, test-time inference offers a promising path forward. I’ll present a particle-based scaling method that avoids the early pruning pitfalls of deterministic search by maintaining diverse candidates and balancing exploration with exploitation. This simple yet efficient approach enables small 1.5B models to outperform GPT-4o in just 4 rollouts and scales 7B models to o1 performance in 32.

Session Recording

July 11th

Gowthami Somepalli ,
Research Scientist @ Adobe Firefly

Secret Life of Noise: Understanding Diffusion Models

This session will explore Diffusion Models. We'll start with the fundamentals of foundational Denoising Diffusion Probabilistic Models (DDPMs) and their more efficient, deterministic successors, Denoising Diffusion Implicit Models (DDIMs). The discussion will also address the critical function of noise schedules in controlling the generation process. Finally, we will introduce flow matching, a modern framework that offers a more direct approach to transforming a simple distribution into complex data

Session Recording

July 11th

Timothy Nguyen,
AI Researcher at Google DeepMind

Understanding Transformers via N-gram Statistics

This session explores how transformer-based language models make predictions by examining their relationship to simple N-gram statistical rules derived from training data. We'll discuss novel findings including: (1) an approximation criterion showing that low-variance predictions tend to be well-described by N-gram rules, (2) how transformers exhibit curriculum learning dynamics by progressing from simple to complex statistical patterns, and (3) a training-intrinsic method for detecting overfitting without holdout data. The work demonstrates that up to 79% of transformer predictions on TinyStories can be matched by optimal N-gram rules, providing quantitative insights into the statistical structure underlying LLM behavior. We'll also examine the implications for understanding memorization, generalization, and the brittleness of language models.

Session Recording

July 12th

Arthur Douillard,
Senior Researcher @ Google DeepMind

Distributed Training in Machine Learning

This talk provides an overview of the landscape of distributed deep learning for LLMs. Due to their sheer size, LLM training must be distributed across multiple GPUs. We’ll first cover the methods sharding the computation across colocated GPUs (such as Fully Sharded Data Parallelism (FSDP), Pipeline and Expert Parallelism (PP & EP). Then, we’ll explore more exploratory methods such as DiLoCo, SWARM, PowerSGD & DeMo, and others which often come with a ML cost but could enable training on GPUs spread across the world.

Slides Link: https://drive.google.com/file/d/13ieV0DLGRtR1RbGT7gUdNHCGB08MSu1r/view

Session Recording

July 14th

Sara Hooker,
Head of Cohere Labs

Research Mentorship Session

This interactive session will help you think about the research process anew, covering such topics as how to find collaborators, consideration for your first research endeavor, and ways to know whether you are working on something that is scientifically meaningful.

Session Recording

July 14th

Madeline Smith and Brittawnya Prince,
Cohere Lab Operations

Exploring the unknown, together: an ML open science social

Take a break from the learning and join us for a social hosted by the Cohere Operations team. This is a chance to connect with fellow researchers from around the world, share experiences, spark new ideas, and explore potential collaborations. We’d love to see you there!

Organizers

Ahmad Mustafa Anis

Cohere Labs Community, Roll.ai

Kanwal Mehreen

Cohere Labs Community, University of British Columbia

Anas Zafar

Cohere Labs Community, University of Texas, MD Anderson Cancer Center

Madeline Smith

Cohere Labs

Brittawnya Prince

Cohere Labs