ECE555: Control of Stochastic Systems

Term: Fall 2020

Prerequisites: ECE 534 (Random Processes) and ECE 515 (Control System Theory and Design)

Instructor: Prof. R. Srikant,

TAs: Joseph Lubars ( and Siddhartha Satpathi (

Office Hours: Joseph: 4-5 Tu, 5-6 Wed; Siddhartha: 5-6 Tu, 4-5 Wed; Srikant: after class

Lectures: 11-12:20 TuTh, via zoom (link is on Compass 2G under announcements)

Topics (Time Permitting):

  • MDPs

  • LQG Problems

  • Controlled Diffusions

Grading :


  • D. P. Bertsekas, Dynamic Programming and Optimal Control, Vols. I and II, Athena Scientific, 2005, 2012

  • D. P. Bertsekas and J. N. Tsitsiklis. Neuro-Dynamic Programming, Athena Scientific, 1996

  • V. Krishnamurthy. Partially Observed Markov Decision Processes. Cambridge University Press, 2016.

  • P. R. Kumar and P. Varaiya. Stochastic Systems: Estimation, Identification and Adaptive Control, SIAM, 2015

  • M. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, 2005

  • S. Ross. Applied Probability Models with Optimization Applications, Dover, 1970

  • S. Ross. Introduction to Stochastic Dynamic Programming, Academic Press, 1983.

  • The notes of Profs. Tamer Basar, Ali Belabbas, Aditya Mahajan, Max Raginsky, Ramon van Handel, and Richard Weber; except for the first one, the rest are available online