First, sorry for my bad english. I am having trouble proving this exercise (it came from some notes I had back in university, I am studying for my masters next year).

Let $X$ be an aperiodic irreducible Markov chain on finite state space $S$. Let $\pi$ be stationary measure. Assume $X$ started at $\pi$. Let $a,b \in S$. Show that:

$\lim_{n \to \infty} \mathbb{P}(X_0=a, X_n=b) = \pi(a)\pi(b)$

I tried many things, including couplings, but cant figure it out. Any tips and help would be great. Thanks!

  • $\begingroup$ Have you computed $\Pr(X_0=a,X_1=b)$ yet? $\endgroup$
    – whuber
    Commented Oct 29, 2020 at 17:36

1 Answer 1


Can't you just write \begin{align} \mathbb P(X_0 = a, X_n = b) &= P(X_0 = a) P(X_n = b \vert X_0 = a) \\ &= \pi(a) P^n_a(b), \end{align} where $P^n_a$ is the $n$-step kernel of the Markov chain started in $a$, and you assume that $X_0$ is drawn from $\pi$. By irreducibility and aperiodicity, this kernel converges to the stationary measure in the limit, so you just get $$ \lim_{n \rightarrow \infty} P^n_a(b) = \pi(b),$$ and the result follows. What am I missing?

  • $\begingroup$ Oh, ok. I am not familiar with Markov kernels (maybe I know them by another name, I studied this in portugues). What exactly is the Markov kernel? The transition matrix? $\endgroup$ Commented Oct 29, 2020 at 11:08
  • $\begingroup$ Yes, exactly. On a discrete space that is exactly what it is. If $K$ is the transition matrix, we just have $P^n_a(b) = K^n(a,b)$. $\endgroup$ Commented Oct 29, 2020 at 11:09
  • $\begingroup$ Great! I think I understand it now. Thank you very much! Now I am also trying to simulate this in R. Any clues how to approach it? I am a bit rusty with all this. (Also, any book recommendation or codes are very welcomed) $\endgroup$ Commented Oct 29, 2020 at 11:12
  • $\begingroup$ You are not gonna go wrong this paper contributed to The R Journal: "Discrete Time Markov Chains with R" by Giorgio Spedicato. $\endgroup$ Commented Oct 29, 2020 at 11:27
  • $\begingroup$ Thank you very much! You were very helpful $\endgroup$ Commented Oct 29, 2020 at 11:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.