Zoom etiquette:

• Keep yourself muted unless I ask you to unmute to ask or answer a question

• To ask a question, do one of the following: (i) raise your hand on zoom, (ii) post a question on chat, or (iii) private message one of the TAs. The TAs will monitor the chat and in case of (i) or (ii), they will interrupt me in case I don't notice the question and then I will ask you to unmute and ask your question. In the case of (iii), the TAs will answer your question as a private message or read your question to me.

The Basics

Dynamic Programming

DP For Stochastic Systems

In-Class Exercise

Infinite-Horizon MDPs

Computational Techniques

Value Iteration for Positive and Negative Costs

Applications

Stochastic Shortest Path Problems

Average-Cost Problems

Continuous-Time MDPs

Semi-Markov Decision Processes

LQG with Imperfect State Information

Ito Calculus and Controlled Diffusions