First, sorry for my bad english. I am having trouble proving this exercise (it came from some notes I had back in university, I am studying for my masters next year).

Let $X$ be an aperiodic irreducible Markov chain on finite state space $S$. Let $\pi$ be stationary measure. Assume $X$ started at $\pi$. Let $a,b \in S$. Show that:

$\lim_{n \to \infty} \mathbb{P}(X_0=a, X_n=b) = \pi(a)\pi(b)$

I tried many things, including couplings, but cant figure it out. Any tips and help would be great. Thanks!

  • $\begingroup$ Have you computed $\Pr(X_0=a,X_1=b)$ yet? $\endgroup$ – whuber Oct 29 at 17:36

Can't you just write \begin{align} \mathbb P(X_0 = a, X_n = b) &= P(X_0 = a) P(X_n = b \vert X_0 = a) \\ &= \pi(a) P^n_a(b), \end{align} where $P^n_a$ is the $n$-step kernel of the Markov chain started in $a$, and you assume that $X_0$ is drawn from $\pi$. By irreducibility and aperiodicity, this kernel converges to the stationary measure in the limit, so you just get $$ \lim_{n \rightarrow \infty} P^n_a(b) = \pi(b),$$ and the result follows. What am I missing?

| cite | improve this answer | |
  • $\begingroup$ Oh, ok. I am not familiar with Markov kernels (maybe I know them by another name, I studied this in portugues). What exactly is the Markov kernel? The transition matrix? $\endgroup$ – Jorge Perez Oct 29 at 11:08
  • $\begingroup$ Yes, exactly. On a discrete space that is exactly what it is. If $K$ is the transition matrix, we just have $P^n_a(b) = K^n(a,b)$. $\endgroup$ – Forgottenscience Oct 29 at 11:09
  • $\begingroup$ Great! I think I understand it now. Thank you very much! Now I am also trying to simulate this in R. Any clues how to approach it? I am a bit rusty with all this. (Also, any book recommendation or codes are very welcomed) $\endgroup$ – Jorge Perez Oct 29 at 11:12
  • $\begingroup$ You are not gonna go wrong this paper contributed to The R Journal: "Discrete Time Markov Chains with R" by Giorgio Spedicato. $\endgroup$ – Forgottenscience Oct 29 at 11:27
  • $\begingroup$ Thank you very much! You were very helpful $\endgroup$ – Jorge Perez Oct 29 at 11:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.