Course Overview

This course provides a rigorous introduction to self-supervised representation learning (SSL), focusing on bridging the gap between empirical deep learning and formal statistical theory. We will explore how models learn representations from unlabeled data by solving pretext tasks. The curriculum is structured around two central themes: first, a comprehensive survey of the algorithmic landscape, examining state-of-the-art frameworks such as Contrastive Methods, Masked Modeling, Video Representation Learning, and Multi-Modal Alignment; and second, an exploration of statistical foundations and recent progress. In the latter half, we will discuss the evolving theoretical underpinnings of these methods, focusing on the role of self-supervised signals, the geometry of latent spaces, and the statistical properties that enable these representations to transfer effectively to downstream tasks. Throughout the course, students are expected to connect algorithms to statistical questions: what self-supervised signal is used, what representation is identifiable, what information is preserved or discarded, how collapse is avoided, and why the representation transfers to downstream tasks.

Textbook

No textbook is required.

Topic Outline

1. Foundation of Representation Learning

2. Contrastive Learning

3. Non-contrastive Siamese Methods

4. Geometry and Theory of Contrastive Representations

5. Self-Supervised Language Modeling

6. Masked Image Modeling and Vision Foundation Model

7. Geometry and Theory of Masked Prediction

8. Predictive Latent-Space Modeling

9. Speech and Video Self-Supervised Learning

10. Multimodal Alignment

Grading

• Three Mini-Projects (60%)

• Final Project (40%)