# Limiting distribution of Markov chain; two approximations

Just for simplicity let's look at Markov states with a finite number of states. So we have a sequence of random variables $(X_t, t\in \mathbb{N})$. Let's assume that there is a steady state distribution $\pi$ to which the chain converges, i.e. as $X_t \to \pi$ when $t \to +\infty$.

Analysing this definition it can be seen that we look at the distribution of the states $X_\infty=\pi$ (a simplified notation is used here). It is known that this limit distribution of the states does not depend on the initial state. So if I simulate the chain $N$ times, starting from $x_1=1$ I well obtain realisations $x_1, x_2^{(k)}, x_3^{(k)}, \dots x_\infty^{(k)}$, for $k=1, 2, \dots N$.

The limiting distribution of the chains is the distribution of $x_\infty^{(k)}, k=1,2, \dots N$ as $N\to\infty$.

In MCMC on the other hand, one does not take a very large number $N \to \infty$ of realisations of the chain, but one (or a limited number of) realisation(s) of a very long chain, i.e. e.g. $x_1, x_2^{(1)}, x_3^{(1)}, \dots x_\infty^{(1)}$ and then $\pi$ is approximated by the relative frequencies of the states in this one long run.

So either I take an infinite number of realisations of the chain and approximate $\pi$ as the distribution of $x_\infty^{(k)}, k=1,2, \dots N$, where $N \to \infty$ or I approximate it as the distribution of the states in one very long realisation i.e. $x_1, x_2^{(1)}, x_3^{(1)}, \dots x_\infty^{(1)}$.

Is there a theorem that states that the latter also converges to $\pi$ ?

Let's assume there is a steady state distribution to which the chain converges, i.e., $X_t \to \pi$ when $t \to +\infty$.
This is the Ergodic theorem that states that if the Markov chain is aperiodic, irreducible and Harris recurrent, then the chain converges to $\pi$ in total variation sense as $t \to \infty$. This result is irrespective of the starting value of the Markov chain. You can find more info in Meyn and Tweedie.