This course teaches an overview of modern optimization methods for applications in economics, machine learning, and data science. In particular, it covers the scalability of algorithms to large datasets, both in theory and in implementation.
Learning objectives
By the end of this course, students will understand the fundamental principles of convex optimization and modern algorithmic techniques used to solve large-scale problems in economics, machine learning, and data science. They will be able to formulate optimization problems, analyze their structure, apply key theoretical tools such as duality and optimality conditions, and implement scalable algorithms including gradient-based and second-order methods. Students will also learn to critically evaluate the performance and suitability of different optimization methods for real-world applications.
Content
Preliminary course structure:
Convex optimization
Optimization problems
Convex sets, convex functions, convex optimization problems
Lagrangian duality
Optimality conditions
Optimization in econometrics, statistics, and machine learning
Algorithms
Gradient descent
Projected gradient descent
Stochastic gradient descent
Newton methods
Structure and indications of the learning and teaching design
The course consists of several lectures that introduce the key theoretical concepts; slides for all lecture material will be provided. Throughout the semester, multiple exercise sheets will be distributed, discussed in class, and assigned as homework. Sample solutions will be made available to support students in their learning.
Prerequisites
A basic background in linear algebra and calculus is required. Elementary knowledge of probability theory is necessary.
Literature
The lecture uses material from the following textbooks (detailed slides will be provided)
Stephen Boyd and Lieven Vandenberghe, Convex Optimization, Cambridge University Press, 2009.
D.P. Bertsekas, Convex Optimization Theory, Athena Scientific, 2009.
Y. Nesterov, Introductory Lectures on Convex Optimization, Springer, 2004
Examination
Written Examination (60 minutes): Friday, June 12, from 15:00–16:00.
Oral Examination (15 minutes): Takes place during the first week of July (approximately four weeks after the course). For internal (HSG) students, the oral exam will be conducted in my office; for external students, it will be conducted via Zoom or Microsoft Teams.
Final Grade: The final course grade is composed of 50% written examination and 50% oral examination.
Lecture Notes
If time allows: Part 11 - Newton Method