Syllabus

1. Week 1 (1/5-9): Vector and Matrices (Chapters 2 and 3 of Chong and Zak)

2. Week 2 (1/12-16): Differentiation (Chapter 5 of Chong and Zak)

• Lecture 3: 5.1, 5.2, 5.3

• Lecture 4: 5.4, 5.5, 5.6

• Supplementary note: Proof of Taylor's theorem and mean value theorem

3. Week 3-4 (1/19-30): Unconstrained & constrained optimization (Chapter 6+19 of Chong and Zak) 

• 1. • First-order necessary condition Lecture 5

     • Second-order necessary condition Lecture 6

     • Second-order sufficient condition Lecture 7

     • Lagrange multiplier Lecture 8

4. Week 5 (2/5): Line Search (Chapter 7 of Chong and Zak) Lecture 9

5. Week 6 (2/9-13): Gradient Descent (Chapter 8 of Chong and Zak)

   • Lecture 10

   • Lecture 11

6. Week 7 (2/16-20): Newton Method (Chapter 9 of Chong and Zak)

   • Lecture 12

   • Lecture 13

7. Week 8-9 (2/23-3/6): Linear/Least-square Regression (Chapter 12 of Chong and Zak)

   • Lecture 14

   • Lecture 15

   • Lecture 16

8.  Week 10 (3/9-13): Neural Networks (Chapter 13 of Chong and Zak)

   • Lecture 17-18