Markov chain

The concept of a Markov chain was developed by Andrey Andreyevich Markov. A Markov chain is a sequence of random variables with the property that it is forgetful of all but its immediate past. For a process Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle {\mathbf {\Phi } }} evolving on a space Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle {\mathsf {X}}} and governed by an overall probability law ${\mathsf {P}}$ to be a time-homogeneous Markov chain there must be a set of "transition probabilities" Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle \{P^{n}(x,A),x\in {\mathsf {X}},A\subset {\mathsf {X}}\}} for appropriate sets $A$ such that for times Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle n,m} in Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle {\mathbb {Z} }_{+}} (Ref. 1 Eq. 1.1)

${\mathsf {P}}(\Phi _{n+m}\in A\vert \Phi _{j},j\leq m;\Phi _{m}=x)=P^{n}(x,A);$

that is Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P^n(x,A)} denotes the probability that a chain at x will be in the set A after n steps, or transitions. The independence of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P^n} on the values of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Phi_j,j \leq m} is the Markov property, and the independence of Failed to parse (Conversion error. Server ("https://wikimedia.org/api/rest_") reported: "Cannot get mml. Server problem."): {\displaystyle P^{n}} and m is the time-homogeneity property.

References

1. S. P. Meyn and R. L. Tweedie "Markov Chains and Stochastic Stability", Springer-Verlag, London (1993)
2. Ruichao Ren and G. Orkoulas "Parallel Markov chain Monte Carlo simulations", Journal of Chemical Physics 126 211102 (2007)