Time: Tue 13:00-16:00

Location: Cluster Bd. (학연산클러스터) R. 507

References: no textbook is required, but following references will be helpful:

• Elements of Statistical Learning, Springer, 2009 -- free PDF available
• Machine Learning, Murphy, MIT Press, 2012
• Infomation theory, inference, and learning algorithms, Mackay, Cambridge, 2017 (9th printing) -- free PDF available
• Foundations of machine learning, Mohri, Rostamizadeh, and Talwalar, MIT Press, 2018 (2nd Ed)
• All of statistics, Wasserman, Springer, 2005
• CS229 Stanford lecture by Andrew Ng

Grading:

• Homework/discussion, Midterm, Final Exam = 30%, 30%, 30%
• Attendance: 10%

Homework

• Homework 1 (due April 2)
• Homework 2 (due April 16)
  • lr_debug.py, data_a.txt, data_b.txt
• Homework 3 (due May 28)

Lecture Notes

Topics will be selected from learning models, learning theory, and large-scale optimization.

• Lecture 1. Introduction (pdf)
• Lecture 2. Supervised Learning (pdf)
  • Linear Regression, Weighted Linear Regression, Perceptron, Exponential Family, Generalized Linear Models
  • Suppl : Linear algebra review (pdf)
• Lecture 3. Generative Models (pdf)
  • Gaussian discriminative analysis, naive Bayes, Laplace smoothing
  • Suppl : Probability theory review (pdf)
  • HW1 is out!
• Lecture 4. Support vector machines (pdf)
  • Suppl: Convex optimization (pdf1, pdf2)
  • Suppl: SMO paper by John Platt 1998 (link), String kernel by Leslie & Kuang 2004 (link)
• Lecture 5. HW1 Discussion & Learning Theory part I (pdf)
  • Suppl: Hoeffding's inequality (pdf)
  • HW2 is out!
• Lecture 6. Learning Theory part 2
• Lecture 7. HW2 Discussion, SMO algorithm, Regularization and Model Selection (pdf)
• Midterm (April 23, in class)
• Lecture 8. EM algorithm (pdf1, pdf2)
  • Bayesian inference (remainder of L7)
  • Midterm review
• Lecture 9. Mixture of Gaussian, Factor Analysis (pdf)
• Lecture 10. PCA (pdf) and ICA (pdf)
  • Eigenfaces
  • HW3 is out!
• Lecture 11. Reinforcement learning and control (pdf)
• Lecture 12. LQR, DDP, and LQG (pdf)
• Lecture 13. POMDP, Policy Search, Reinforce, Pegasus
• Final Exam (June 11, in class)