Course Description & Basic Information

Professor: Elad Hazan

The course address optimization problems that arise in machine learning, as well as efficient algorithms to solve them. The course is proof-based, and contains both theory and applied exercises (choice given).

Topic covered:

• Introduction to convex analysis
• first-order methods, convergence analysis
• generalization and regret minimization
• regularization
• gradient descent++:
  • acceleration
  • variance reduction
  • adaptive preconditioning
• 2nd order methods in linear time
• projection-free methods and the Frank-Wolfe algorithm
• zero-order optimization, convex bandit optimization
• optimization for deep learning: large scale non-convex optimization

Lectures

Tuesdays 10:00-12:20, in Computer Science Building Rm 402

Tuesdays 09:00-10:00, self-study in COS 402 and lecture preparation / office hours (optional)

Professors' office hours: Mon 9-10am in COS 409 or COS 402