lecture topics and notes

Resources for typing notes: def.tex scribe template file

Joint text of (instructor written) lecture notes

(this file will be updated throughout the course)

• Lecture 1: Introduction to optimization for machine learning. lecture notes: above text.

notes: chapters 1,2 in text

• Lecture 2: Basic concepts in analysis and optimization . Gradient descent and Stochastic Gradient Descent for ML.

notes: chapters 3 in text, and scribe nots by Paula Gradu

• Lecture 3: Generalization and regret

notes: chapter 4 in text, and scribe notes by Lawrence Thul, and by Diana Cai

• Lecture 4: Regularization and mirrored descent

notes: chapter 5 in text, scribe notes by Geoffrey Roeder

• Lecture 5: Adaptive Regularization

notes: chapter 6 in text, and scribe notes by Bill Huang , and more notes by Fangyin Wei

• Lecture 6: Variance Reduction

notes: chapter 7 in text, and scribe notes by Arushi Gupta, and more by Sulin Liu

• Lecture 7: Projection-free algorithms and the Frank-Wolfe method

notes: chapter 8 in text, and scribe notes by Pranay Manocha

• Lecture 8: Nesterov acceleration

• Lecture 9: Second order methods for machine learning

notes: chapter 9 in text, and scrive notes by Mayee Chen, Ethan Tseng, Edgar Minasyan

• Lecture 10: Hyperparameter optimization (part I) & ask me anything (part II)