I was reading my notes, it says that:

''A sufficient condition $p(\theta)$ to be the stationary distribution is the reversibility: $$\sum_{\theta}p(\theta)p(\theta^*|\theta)=\sum_{\theta}p(\theta^*)p(\theta|\theta^*)=p(\theta^*)\sum_{\theta}p(\theta|\theta^*)=p(\theta^*).''$$ I was wondering what does the reversibility mean here, since this expression is trivially true for any $p(\theta)$. And why the reversibility is a sufficient condition that makes $p(\theta)$ a stationary distribution? Since by definition $p(\theta)$ needs to satisfy $p(\theta)=Ap(\theta)$ where $A$ is a transition matrix (Markov Chain).


A discrete Markov chain $\theta_1, \theta_2, \ldots$ with transition pmf $k(\theta_{t+1} \mid \theta_t)$ is reversible if joint pmfs for two consecutive time points are "symmetric", or in other words $$ p(\theta_{t+1} = a, \theta_{t} =b) = p(\theta_{t+1} = b, \theta_{t} =a) $$ for any $a,b$ in the state space. You can think of it as the probability of starting in $b$ and going to $a$ is the same as the probability of starting in $a$ and going to $b$. The order doesn't matter.

When we write it with the transition pmf and stationary distribution $\pi_{\theta}(\cdot )$ we get something closer to your notation: $$ k(a \mid b) \pi(b) = k(b \mid a)\pi(a). $$

Any measure $\pi$ that satisfies reversibility must also satisfy stationarity. This can be seen if we sum both sides over all possible values of $a$: $$ \pi(b) = \sum_{a} k(b \mid a) \pi(a). $$

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.