Given a Markov chain, an absorbing state is a state which, when reached, cannot be left. For instance, the population of a species can be modelled as a Markov chain evolving in {0,1,2,...}, so X_n=i means that the population is i at time n. When the population reaches 0, the species is extinct and hence no new individuals can be born, hence 0 is an absorbing state.
The stationary distribution for an absorbed Markov chain is simply the distribution with mass 1 on the absorbing state. Convergence of the Markov chain to its stationary distribution therefore only tells you that it will eventually be absorbed. One is instead interested in the long time limits of
P(X_n=i | not absorbed by time n) as n -> infinity.
These limits are distributions called quasi-stationary distributions. These have found application in a wide variety of applications, from ecology and population genetics to nuclear reactor physics for example, where a certain quasi-stationary distribution plays a central role in safely controlling nuclear reactors. The principal goal is to understand when these limits exist, and how to characterise them.
In order to sample from the stationary distribution for a given Markov chain, one can simply run the Markov chain for a long time. However, if one does that with an absorbed Markov chain, it is likely that it will have been absorbed by a sufficiently large time $n$. Sampling from a quasi-stationary distribution by naively running an absorbed Markov chain for a long time and taking those samples which aren't absorbed therefore doesn't work, because almost all of the samples will be absorbed. We will learn about two methods for getting around this, one based on a system of particles and one based on a random walker jumping onto its own history.
Group project
We will gain a thorough understanding of:
Continuous-time Markov chains.
Basic results on when an absorbed Markov chain converges to a quasi-stationary distribution, and how to characterise them.
Two methods for sampling from a quasi-stationary distribution: the Fleming-Viot particle system and reinforced processes.
Mode of operation and evidence of learning
The first part of the project will be directed reading on the definitions and background of continuous time, discrete space Markov chains.
We will then engage in directed reading on the theory of absorbed Markov chains, and its applications.
We will concurrently study the Fleming-Viot particle system and reinforced processes.
We will code up realisation of these numerical algorithm and compare their performance.
The survey by Sylvie Meleard and Denis Villemonais, https://arxiv.org/pdf/1112.4732, provides an excellent exposition of the content we will cover.
For background on continuous time, discrete space Markov chains, I suggest the lecture notes https://www.statslab.cam.ac.uk/~ps422/notes-new.pdf.