I have a transition matrix P,i don't understand how my teacher explains properties of the states without any calculation. In example :

$$P= \begin{array}{ccc} \frac12 & \frac14 & \frac14 \\ 0 & \frac12 & \frac12 \\ 0 & 0 & 1 \end{array}$$ How can i say that the first and second states are recurrent? Is this chain irreducible because the third state doesn't communicates with other states? And how can i understand the period of a state ?


It seems there has been some miss understanding, since all the statements you made are wrong for the given Markov-chain.

  • You can not say that the first and second states are recurrent because the third state is absorbing (i.e. once you are in the third state you will never leave it and therefore you'll never return to state one or two).
  • The chain is not irreducible. A Markov-chain is called irreducible if all states form one communicating class (i.e. every state is reachable from every other state, which is not the case here).
  • The period of a state is defined as $k$, if that state can only be reached every $k$-th step. For example if you have two states that flip to the other state with probability one and and you start at $t=0$ in state one, then at time $t=1$ you must be in state two for sure, then at $t=2$ you must be in state one again, at $t=3$ in state two again. So state one is visited at $t=0, 2, 4, ...$, therefore it has period $2$.

All of these concepts are very well explained on the Wikipedia article on markov chains. I highly recommend you read the corresponding sections.

  • $\begingroup$ so can i say for sure that if exists a absorbing state other states have to be transient? $\endgroup$
    – Federico
    Jan 19 '16 at 17:09
  • $\begingroup$ Yes. If they communicate with that absorbing state, eventually the process will leave the other states and will get stuck in the absorbing state and therefore never return again. $\endgroup$
    – Denwid
    Jan 19 '16 at 17:17

In order to understand recurrence, transience, non-return state, and absorbing state, we don't require the actual transition probabilities of a Markov Chain, if the states of the chain can be accommodated in state transition diagram. For example, to understand the nature of the states of above Markov Chain, the given transition matrix can be equivalently be represented as

\begin{equation*} P = \left(\begin{array}{ccc} * & * & *\\ 0 & * & *\\ 0 & 0 & *\\ \end{array}\right) \end{equation*}

where a * stands for positive probability for that transition.

Now, draw the state transition diagram of the Markov Chain.

enter image description here

There are 3 communicating classes, here: {1}, {2} and {3}. Now identify which of these classes are closed communicating classes and non-closed communicating classes.

Consider class {1}. State 1 communicates with itself. However, an escape is possible to state 2 or state 3. Hence, it is a non-closed communicating class. States in a non-closed communicating classes become transient states.

Class {2} can be interpreted in a similar manner.

State 3 communicates with itself and all the edges are into the state 3. Hence, state 3 itself forms a closed-communicating class. States in a closed communicating classes become recurrent states. As there is only one state in this communicating class, the state is called an absorbing state. In a finite Markov Chain, there must be at least one recurrent state. As all the states do not belong to a single communicating class, the given chain is not irreducible.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.