Markov chain

The concept of a Markov chain was developed by Andrey Andreyevich Markov. A Markov chain is a sequence of random variables with the property that it is forgetful of all but its immediate past. For a process ${\mathbf {\Phi } }$ evolving on a space ${\mathsf {X}}$ and governed by an overall probability law Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle {\mathsf {P}}} to be a time-homogeneous Markov chain there must be a set of "transition probabilities" $\{P^{n}(x,A),x\in {\mathsf {X}},A\subset {\mathsf {X}}\}$ for appropriate sets $A$ such that for times $n,m$ in Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {\mathbb Z}_+} (Ref. 1 Eq. 1.1)

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle {\mathsf P} (\Phi_{n+m} \in A \vert \Phi_j,j \leq m; \Phi_m =x)= P^n(x,A);}

that is Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P^n(x,A)} denotes the probability that a chain at x will be in the set A after n steps, or transitions. The independence of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P^n} on the values of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Phi_j,j \leq m} is the Markov property, and the independence of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P^n} and m is the time-homogeneity property.

References

1. S. P. Meyn and R. L. Tweedie "Markov Chains and Stochastic Stability", Springer-Verlag, London (1993)
2. Ruichao Ren and G. Orkoulas "Parallel Markov chain Monte Carlo simulations", Journal of Chemical Physics 126 211102 (2007)