Optimization Theory and Applications

• Textbook & References: 

  • 1.  Algorithms for Optimization

by Mykel J. Kochenderfer and Tim A. Wheeler.
The MIT Press, 2/e Draft in 2025. [Link]

• • 2. An Introduction to Optimization: With Applications to Machine Learning
    by Edwin K. P. Chong, Wu-Sheng Lu, Stanislaw H. Zak.
    Wiley, 2023. [Link] 

• Main Topics:

  • 1. Unconstrained Optimization

  • 2. Optimization in Machine Learning & Deep Learning 

    • References: 

      • Deep Learning by Ian Goodfellow, Yoshua Bengio & Aaron Courville.
        Chapter 8: Optimization for Training Deep Models [Link]

      • Deep Learning: Foundations and Concepts by Christopher M. Bishop & Hugh Bishop.
        Chapter 7: Gradient Descent [Link]

      • Dive into Deep Learning by Aston Zhang, Zachary C. Lipton, Mu Li, & Alexander J. Smola.
        Chapter 12: Optimization Algorithms [Link]

  • 3. Linear Programming

  • 4. Nonlinear Constrained Optimization

• Evaluation:

Class participation15%, Homework & Project 50%,
Midterm and Final Exam 35%