These are notes that I use to prepare for class. These are not intended to be official lecture notes. In particular, I don't proof-read them carefully. However, if you do find typos, please let me know and I will correct them. You are expected to come to class and take notes. Please do not ask me to put up the notes before lectures, I may not always be able to do this.

• Markov Chains

• MDPs

• Computational Techniques

• Tabular TD Learning

• TD Learning with Linear Function Approximation

• Q-Learning, Tabular and LFA

• Descent Lemma, Gradient Descent

• Descent lemma, Convergence of Gradient Descent: Fixed Step size

• SGD, Mirror Descent

• Policy Gradient Algorithms

• NPG

• Multi-armed Bandits

• UCB-RL

These are typed notes based on the lectures, courtesy of Dr. Mehrdad Moharrami, but there may be some differences from the handwritten version

• Markov Chains

• MDPs

• Computational Techniques

• Tabular TD Learning