Online Decision Making: The utility of Fluid Heuristics & Optimism in the face of uncertainty

Topics to be covered this six week course

1. An introduction to Markov Decision Processes

2. Dynamic Resource-Constrained Reward Collection Problems

3. Online Allocation problems

4. Multi-Armed Bandits & Bandits with Knapsacks

5. Contextual Bandits & Contextual Bandits with Knapsacks

6. Operationalising the principle of optimism in broader Reinforcement Learning problems

References:

1. Reinforcement Learning: Theory and Algorithms 

Monograph by Alekh Agarwal, Nan Jiang, Sham Kakade, Wen Sun

2. Survey of Dynamic Resource-Constrained Reward Collection Problems: Unified Model and Analysis

Journal article by Santiago R. Balseiro, Omar Besbes, Dana Pizarro

Operations Research (In Press)

3. The best of many worlds: Dual Mirror Descent for Online Allocation Problems

Journal article by Santiago R. Balseiro, Haihao Lu, Vahab Mirrokni

Operations Research 71(1), pp. 101-119

4. Foundations of Reinforcement Learning and Interactive Decision Making

Monograph by Dylan J. Foster and Alexander Rakhlin

Lecture slides

Week 2: Dynamic Resource-Constrained Reward-Collection Problems: Introduction, Examples, Fluid approximation, and the CE heuristic (slides)

Week 3: The fluid approximation and the CE heuristic in problems (slides)

Please note: These slides are just used as teaching aid for this class. They are not meant to be reproduced