ECE555: Control of Stochastic Systems

Term: Fall 2020

Prerequisites: ECE 534 (Random Processes) and ECE 515 (Control System Theory and Design)

Instructor: Prof. R. Srikant, rsrikant@illinois.edu

TAs: Joseph Lubars (lubars2@illinois.edu) and Siddhartha Satpathi (ssatpth2@illinois.edu)

Office Hours: Joseph: 4-5 Tu, 5-6 Wed; Siddhartha: 5-6 Tu, 4-5 Wed; Srikant: after class

Lectures: 11-12:20 TuTh, via zoom (link is on Compass 2G under announcements)

Topics (Time Permitting):

  • MDPs

  • LQG Problems

  • Controlled Diffusions

Grading :

  • Homework: 60%; Midterm Exam (Nov. 5): 20%; Project: 20%

References:

  • D. P. Bertsekas, Dynamic Programming and Optimal Control, Vols. I and II, Athena Scientific, 2005, 2012

  • D. P. Bertsekas and J. N. Tsitsiklis. Neuro-Dynamic Programming, Athena Scientific, 1996

  • M. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, 2005

  • S. Ross. Applied Probability Models with Optimization Applications, Dover, 1970

  • P. R. Kumar and P. Varaiya. Stochastic Systems: Estimation, Identification and Adaptive Control, SIAM, 2015

  • The notes of Profs. Tamer Basar, Ali Belabbas, Max Raginsky, Ramon van Handel, and Richard Weber, the last four are available online