Chapters 2 - 5 of our forthcoming book, Introduction to Markov Decision Processes are available for downloading.
Introduction to Decision Analysis and MDPs
