Finite Markov Chains

Diffusion of Gases, Genetics, and Random Walks on a Graph

With Thomas Schellhous and Sean Watson

Markov chains are stochastic processes in which a system transitions between given states and each transition depends solely on the current state, without regard to past results. This Markov property makes them ideal statistical models for many real-world processes. They can be used to describe and predict the behavior of wide variety of phenomena, from sports to finance to medicine and even the internet, and have been found in popular culture through their use in text generators and comics. We will focus our attention on finite state Markov chains with discrete time transitions, first studying their general properties and then focusing on the two separate cases of absorbing and ergodic Markov chains. Applications we will consider include the Ehrenfest urn model as a simple model of gas diffusion, predicting the effects of inbreeding, and random walks on graphs, as well as more complex applications such as indexing the internet and predicting stock prices. This project should be accessible to all students who have taken one quarter of linear algebra. Knowledge of basic probability theory will be useful, but not required.