From high stake stock trading to safely driving to simply figuring out your shopping list for the month, modern life is filled with situations in which we handle, manage, or control our assets, driving, and actions in face of various forms of uncertainty. 
In this course, we consider a (Baysian and) Markov model of such uncertain systems. We first start with the simple analysis of discrete time Markov processes with countable state space, also known as Markov chains, and then proceed to the topics of control, optimization, and estimation.