Markov chain

The concept of a Markov chain was developed by Andrey Andreyevich Markov. A Markov chain is a sequence of random variables with the property that it is forgetful of all but its immediate past. For a process $\mathrm{\Phi }$ evolving on a space $\mathsf{X}$ and governed by an overall probability law $\mathsf{P}$ to be a time-homogeneous Markov chain there must be a set of "transition probabilities" $\{P^n(x,A),x\in \mathsf{X},A\subset \mathsf{X}\}$ for appropriate sets $A$ such that for times $n,m$ in ${\mathbb{Z}}_{+}$ (Ref. 1 Eq. 1.1)

$\mathsf{P}({\mathrm{\Phi }}_{n+m}\in A|{\mathrm{\Phi }}_j,j\le m;{\mathrm{\Phi }}_m=x)=P^n(x,A);$

that is $P^n(x,A)$ denotes the probability that a chain at x will be in the set A after n steps, or transitions. The independence of $P^n$ on the values of ${\mathrm{\Phi }}_j,j\le m$ is the Markov property, and the independence of $P^n$ and m is the time-homogeneity property.

References

1. S. P. Meyn and R. L. Tweedie "Markov Chains and Stochastic Stability", Springer-Verlag, London (1993)
2. Ruichao Ren and G. Orkoulas "Parallel Markov chain Monte Carlo simulations", Journal of Chemical Physics 126 211102 (2007)