Introduction to Markov Decision Processes
By Martin L. Puterman and Timothy C. Y. Chan
We are currently writing a book that will be an accessible and up to date introduction to Markov decision processes (MDPs). Our target audiences include undergraduate students, graduate students and self-directed learners looking for a structured introduction that covers foundations, algorithms, and applications.
Drafts of certain chapters will be posted here as they become available. We welcome all feedback and suggestions. We believe Chapters 2-5 provide the necessary material for a course on MDPs.
This material will be published by Cambridge University Press as Introduction to Markov Decision Processes by Martin L. Puterman and Timothy C. Y. Chan. This pre-publication version of the following chapters is free to view and download for personal use only. Not for redistribution, resale, or use in derivative works. ©Martin L. Puterman and Timothy C. Y. Chan, 2023.
Chapter 2: Model Foundations
Chapter 3:Examples and Applications
Chapter 4: Finite Horizon Models
Chapter 5: Infinite Horizon Discounted MDPs