Seminar: Machine Learning and Optimization

This is a seminar series, whose topic changes every term.

Please register in ZEuS for the Seminar 

Current Seminar:

Summer 2024: Introduction to Online Nonstochastic Control

 Based on the book "Introduction to Online Nonstochastic Control", Elad Hazan, 2022 (link)

Control theory is the engineering science of manipulating physical systems to achieve desired functionality. Its history is centuries old, and it spans rich areas from the mathematical sciences. A brief list includes differential equations, operator theory and linear algebra, functional analysis, harmonic analysis, and more. However, it mostly does not concern modern algorithmic theory and computational complexity from computer science. The advance of the deep learning revolution has brought to light the power and robustness of algorithms that are based on gradient methods.

This Seminar concerns this exact observation: how can we exploit the differentiable structure in natural physical environments to create efficient and robust gradient-based algorithms for control?

The answer we give is not to apply gradient descent to any existing algorithm. Rather, we rethink the paradigm of control from the start. What can be achieved in a given dynamical system, whether known or unknown, fully observed or not, linear or non-linear, and without assumptions on the noise sequence? This is a deep and difficult question that we fall short of answering. We do, however, propose a different way of answering this question: we define the control problem from the point of view of an online player in a repeated game. This gives rise to a simple but powerful framework called nonstochastic control theory. The framework is a natural fit to apply techniques from online convex optimization, resulting in new, efficient, gradient based control methods.

This seminar will provide an introduction to the concept or Online Nonstochastic Control, which requires a solid mathematical concepts related to convex optimization and Markov decision processes.


We will roughly cover the following topics:

This seminar will be based on the following book:

"Introduction to Online Nonstochastic Control",  Elad Hazan, 2022 (link)


Prerequisites

Logistics

Schedule 

Previous Seminars:

Topic for Summer 2023: Introduction to Online  Convex Optimization

Based on Introduction to Online Convex Optimization, Elad Hazan, 2021 (link)


Online Convex Optimization (OCO) considers optimization as a process. In many practical applications, the environment is so complex that it is not feasible to lay out a comprehensive theoretical model and use classical algorithmic theory and mathematical optimization. It is necessary, as well as beneficial, to take a robust approach, by applying an optimization method that learns as more aspects of the problem are observed. This view of optimization as a process has become prominent in various fields, which has led to spectacular successes in modeling and systems that are now part of our daily lives.

The growing body of literature of machine learning, statistics, decision science, and mathematical optimization blurs the classical distinctions between deterministic modeling, stochastic modeling, and optimization methodology. This seminar continues this trend by studying a prominent optimization framework whose precise location in the mathematical sciences is unclear: the framework of online convex optimization (OCO), which was first defined in the machine learning literature. The metric of success is borrowed from game theory, and the framework is closely tied to statistical learning theory and convex optimization.


This seminar will provide an introduction to the concept or Online Convex Optimization, which requires a solid mathematical concepts related to convex optimization and Markov decision processes.

We will roughly cover the following topics:


This seminar will be based on the following book:

1) Introduction to Online Convex Optimization, Elad Hazan, 2021 (link)


Prerequisites

Logistics


Schedule 

Topic for Winter 2022/2023: An Introduction to Reinforcement Learning

To define reinforcement learning (RL) it is first necessary to define automatic control. Examples in your everyday life may include the cruise control in your car, the thermostat in your air-conditioning, refrigerator and water heater, and the decision making rules in a modern clothes dryer. There are sensors that gather data, a computer to take the data to understand the state of the “world” (is the car traveling at the right speed? Are the towels still damp?), and based on these measurements an algorithm powered by the computer spits out commands to adjust whatever needs to be adjusted: throttle, fan speed, heating coil current, or ... More exciting examples include space rockets, artificial organs, and microscopic robots to perform surgery. The dream of RL is automatic control that is truly automatic; without any knowledge of physics or biology or medicine, an RL algorithm tunes itself to become a super controller: the smoothestride into space, and the most expert micro-surgeon!


This Seminar will provide an introduction to the concept or RL, which requires a solid mathematical understanding of the underlying optimal control problem and Markov decision processes.

We will roughly cover the following topics:


This seminar will be based on the following two books:

1) Control Systems & Reinforcement Learning, Sean Meyn, 2022 (link)

2) Reinforcement Learning: An Introduction, Richard Sutton & Andrew Barto, 2015 (link)


Prerequisites

Logistics


Schedule (tbd)

Topic for Summer 2022: Fairness and Machine Learning - Limitations and Opportunities

In the last years, Machine Learning has made impressive progress and has entered socio-technical systems such as video surveillance and automated resume screening. At the same time, there has been increased public concern about the impact of digital technology on society. These two trends have led to the emergence of fairness, accountability and transparency in socio-technical system as a rapidly growing research field.


This Seminar aims to address the issue of fairness in algorithmic decision making in a principled manner. While we will not have an all-encoompassing formal definition of fairness, we will study current practices of Machine Learning aiming to achieve specific notations of fairness. 

We will roughly cover the following topics:

The textbook we will use for this seminar is FAIRNESS AND MACHINE LEARNING Limitations and Opportunities by Solon Barocas, Moritz Hardt, Arvind Narayanan

Logistics

Schedule