ECE7995: Online Decision Making

(Fall 2021)

  • Instructor: Xingyu Zhou, Wayne State Unviersity.

  • Contents: Introduction to mathematical foundations of online learning, including online convex optimization, bandits, and reinforcement learning, mostly about theorems and proofs. Meanwhile, it also covers basic Python implementations of commonly used algorithms.

  • Grades: 3 homework assignments and 1 final project

Lecture Notes (* notes may contain various typos or errors. Feel free to let me know:-))

Part I -- Online Convex Optimization

Part II -- Bandits

Part III -- Reinforcement Learning

Final Review


Reference Materials

Online Convex Optimization

  • Online Learning and Online Convex Optimization, by Shai Shalev-Shwartz [online access]

  • A Modern Introduction to Online Learning, by Francesco Orabona [online access]

Bandits Learning

Reinforcement Learning

  • Reinforcement Learning: Theory and Algorithms, by Alekh Agarwal Nan Jiang Sham M. Kakade Wen Sun [online access]

  • Reinforcement Learning: An Introduction, by Richard S. Sutton and Andrew G. Barto [online access]

Reference Courses