3
$\begingroup$

I am recently studying Markov Chain. However, I encountered one question regarding the difference between Ergodic Markov chain and time reversible Markov chain. My question is:

What is the difference between time reversible Markov chain and Ergodic Markov chain?

$\endgroup$
5
$\begingroup$

If I take the definition of David Aldous and Jim Fill, a finite state space Markov chain is time-reversible if it satisfies the detailed balance equation $$\pi_i\,p_{ij}=\pi_j\,p_{ji}$$ where the $p_{ij}$'s are the terms of the Markov transition matrix and the $\pi_i$'s are the terms of a probability distribution. Then, by summing both sides of the equation in $i$, we derive that $$\sum_{i=1}^N \pi_i\,p_{ij}=\sum_{i=1}^N \pi_j\,p_{ji}=\pi_j\,\sum_{i=1}^N p_{ji}=\pi_j$$ which implies that $(\pi_1,\ldots,\pi_N)$ is a stationary distribution for the Markov transition. If the chain is assumed to be irreducible, then the stationary distribution is unique. And the Markov chain is then ergodic if it is aperiodic.

The converse is not true, in that there exist non-reversible ergodic Markov chains. An example is provided by the Gibbs sampler associated with a vector $(X_,X_2,X_3)$ and a stationary distribution $P(x_1,x_2,x_3)$. Considering the transition from $\mathbf{X}^t$ to $\mathbf{X}^{t+1}$

  1. Generate $X_1^{t+1}\sim P(x_1|X_2^t,X_3^t)$
  2. Generate $X_2^{t+1}\sim P(x_1|X_1^{t+1},X_3^t)$
  3. Generate $X_3^{t+1}\sim P(x_1|X_1^{t+1},X_2^{t+1})$

is not time-reversible, but if the three conditional distributions have no restriction on their support, the resulting Markov chain is ergodic with distribution $P$.

$\endgroup$
  • $\begingroup$ Hi @Xi'an, Thanks for the clarification. Can you be more specific on the reversibility on the Gibbs sampling? I am still not sure about the reason why it is not time-reversible. Can you make an example, say Gibbs sampling from a multi-nomial distribution? $\endgroup$ – Rax Yao Oct 20 '16 at 5:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.