I'm having trouble with understanding this theory that states that every irreducible chain (we are talking about finite states Markov Chain) has a unique stationary distribution if and only if all the states are positive recurrent. Nevertheless, consider the transition matrix

$M= \begin{bmatrix} 0.5 & 0.5 & 0 & 0\\ 0 & 0.5 & 0.5 & 0 & \\ 0 & 0 & 0.5 & 0.5\\ 0 & 0 & 0 & 1 \end{bmatrix}$

The only recurrent state for this matrix is state 3 (The states are 0,1,2,3) Nevertheless, this matrix has a unique stationary distribution $(0,0,0,1)$ upon solving the equation $\pi * M = \pi$ despite not all states are positiver recurrent. Can somebody explain to me if there is a lapse in my theory?

Thank you

  • $\begingroup$ I haven't looked up all the definitions in a while (which is why this is only a comment) but does it hinge on the "unique" part? Many different matrices that had a similar "black hole" (state you couldn't leave) would all have the SAME stationary distribution. $\endgroup$ – one_observation Nov 27 '16 at 17:16
  • 1
    $\begingroup$ This chain is not irreducible, so the theorem does not apply. $\endgroup$ – whuber Nov 27 '16 at 17:28
  • $\begingroup$ Okay so you mean it is because the chain is not irreducible in the first place, and the theorem only applies if the chain is irreducible? So in this case is there any connection between having a stationary distribution with the number of closed intercommunicating states within the chain itself? $\endgroup$ – chrishendra93 Nov 28 '16 at 1:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.