# Markov Chain Limit Proof

First, sorry for my bad english. I am having trouble proving this exercise (it came from some notes I had back in university, I am studying for my masters next year).

Let $$X$$ be an aperiodic irreducible Markov chain on finite state space $$S$$. Let $$\pi$$ be stationary measure. Assume $$X$$ started at $$\pi$$. Let $$a,b \in S$$. Show that:

$$\lim_{n \to \infty} \mathbb{P}(X_0=a, X_n=b) = \pi(a)\pi(b)$$

I tried many things, including couplings, but cant figure it out. Any tips and help would be great. Thanks!

• Have you computed $\Pr(X_0=a,X_1=b)$ yet?
– whuber
Commented Oct 29, 2020 at 17:36

Can't you just write \begin{align} \mathbb P(X_0 = a, X_n = b) &= P(X_0 = a) P(X_n = b \vert X_0 = a) \\ &= \pi(a) P^n_a(b), \end{align} where $$P^n_a$$ is the $$n$$-step kernel of the Markov chain started in $$a$$, and you assume that $$X_0$$ is drawn from $$\pi$$. By irreducibility and aperiodicity, this kernel converges to the stationary measure in the limit, so you just get $$\lim_{n \rightarrow \infty} P^n_a(b) = \pi(b),$$ and the result follows. What am I missing?
• Yes, exactly. On a discrete space that is exactly what it is. If $K$ is the transition matrix, we just have $P^n_a(b) = K^n(a,b)$. Commented Oct 29, 2020 at 11:09