COS 598D: Optimization for Machine Learning

Course Description & Basic Information

Professor: Elad Hazan

The course address optimization problems that arise in machine learning, as well as efficient algorithms to solve them. The course is proof-based, and contains both theory and applied exercises (choice given).

Topic covered:

  • Introduction to convex analysis
  • first-order methods, convergence analysis
  • generalization and regret minimization
  • regularization
  • gradient descent++:
    • acceleration
    • variance reduction
    • adaptive preconditioning
  • 2nd order methods in linear time
  • projection-free methods and the Frank-Wolfe algorithm
  • zero-order optimization, convex bandit optimization
  • optimization for deep learning: large scale non-convex optimization


Tuesdays 10:00-12:20, in Computer Science Building Rm 402

Tuesdays 09:00-10:00, self-study in COS 402 and lecture preparation / office hours (optional)

Professors' office hours: Mon 9-10am in COS 409 or COS 402