Computational Control Theory
Course Description & Basic Information
A theoretical introduction to control theory and reinforcement learning, focusing on continuous state spaces and applications from the physical world and robotics. We emphasize computationally efficient algorithms and provable bounds. A special focus will be the new methodologies of non-stochastic control and regret minimization in RL. We will compare and contrast with classical methodology in the field.
The course exercises and projects will require coding in python.
The course is open to all students, but a strong mathematical background is required.
Textbook and readings
Textbooks:
1. Dynamic Programming and Optimal Control, by Dimitri Bertsekas
2. Underactuated Robotics by Russ Tedrake
3. Reinforcement learning: an introduction, by Richard S. Sutton, Andrew G. Barto
4. Reinforcement Learning: Theory and Algorithms (draft), by Alekh Agarwal, Nan Jiang, Sham M. Kakade
5. Bandit Algorithms, by Tor Lattimore and Csaba Szepesvari
Supplementary textbooks:
1. Introduction to Online Convex Optimization, by E. Hazan, available here
2. Boosting: Foundations and Algorithms, by R. E. Schapire and Y. Freund
3. Lecture Notes: Optimization for Machine Learning, by E. Hazan, available here
Administrative Information
Lectures: Wed, 13:30-16:30 in TBD
Professor: Elad Hazan, COS building 409, Office hours: TBD , or by appointment.
Teaching Assistants:
Xinyi Chen, Office hours: by appointment
Paula Gradu, Office hours: by appointment
Piazza channel:
Requirements: This is a graduate-level course that requires significant mathematical background.
Required background: probability, discrete math, calculus, analysis, linear algebra, algorithms and data structures, theory of computation / complexity theory
Recommended: linear programming, mathematical optimization, game theory
Attendance: Attendance is required at all lectures. Class participation will be included in the final grade.
Grading and collaboration
Grading: TBD in class 1
Scribe signup sheet, scribe template