Teaching
“A teacher is a person who passes down the Dao, imparts professional knowledge and resolves doubt." - Han Y
"师者，所以传道受业解惑也。" - 韩愈
Deep Learning Materials
Understanding Deep Learning by Simon Prince
Machine Learning 2021 by Hung-yi Lee (focus on general deep learning)
Machine Learning 2023 by Hung-yi Lee (focus on generative AI)
Generative AI 2024 by Hung-yi Lee
Transformers at Hugginface
STAD80: Analysis of Big Data
The prerequisites for STAD80 are STAC58H3, STAC67H3, and CSCC11H3. Please plan ahead if you want to take this course. (I am on leave from 2023 to 2024. So this course will not be offered in 2024. Please plan ahead.)
There won't be textbooks for this course.
This course's materials evolve from year to year.
This course has a heavier workload than others.
This course uses independent research projects to evaluate the student performance in part.
All works must be typed, using Latex, and submitted at Quercus.
No late work.
This course is by no means a bird course. If you want an easier one, take something else.
Do not cheat in this class, just drop it and take an easier one. No one cares whether you took this course or not.
Topics for Winter 2023
General introduction and fundamental concepts
Supervised learning: Regression
Supervised learning: Classification
High dimensional statistics
Neural networks: Expressivity and optimization
No course (midterm exam)
Architecture design: CNN and self attention
Seq2seq I: Transformer, bert, and chatGPT
Seq2seq II: Transformer, bert, and chatGPT
Generative models I: linear, GAN, VAE, and diffusion models
Generative models II: linear, GAN, VAE, and diffusion models
Good Friday
Other potential topics: Self-supervised learning (contrastive- and masked- based), transfer learning (representation, fine and prompt tuning, lifelong, negative transfers), dependence learning (Gaussian graphical models, random dot graphs, and GNNs), network compression, ensemble learning (bagging, stacking, boosting, and meta learning).
Independent Research Projects
Unfortunately, I no longer have time to supervise independent research projects. If you think you have a compelling story though, for example, you have published papers in top ML/Stats venues or you are an ICPC world finalist, feel free to reach out. Otherwise, please consider to take STAD80 instead.
Deep Learning Theory
I will have a new semester-long course on deep learning theory (DLT) in the year of 2023--2024. Stay tuned and check back later.
Courses Taught
Summer 2021, 2022: Deep Learning Theory
Winter 2021: STAC58 Statistical Inference
Winter 2020, 2021, 2022: STAD80 Analysis of Big Data
Fall 2019: STA4527 Random Matrix Theory and Its Applications
Summer 2019: High Dimensional Statistics
Winter 2018, 2019, 2020: STAC63 Probability Models
Winter 2018, 2019: STA3000 Advanced Theory of Statistics