Lets say there is a Markov chain whose transition matrix is defined as follows

$$ P = \left( \begin{array}{cccc} 0.5 & 0.5 & 0 & 0 \\ 0 & 0.25 & 0 & 0.75 \\ 0 & 1 & 0 & 0 \\ 0 & 1 & 0 & 0 \end{array} \right) $$

Now I've been told that communicating classes are those subsets of states communicate with each other, i.e. $P_{ij} > 0$ and $P_{ji} > 0$.

In the example above 1 communicates only with itself, 2 and 4 communicate with each other, but state 3 doesn't communicate with any other state, even itself.

So the communicating classes are {1} and {2,4}. But is {3} a such a class? It seems odd for it not to be, but it doesn't seem to match the definition as I've seen it. Presumably the chain is still reducible?

Edit: Fixed an issue where initially the top row of P failed to sum to one


no it doesn't, it doesn't match the definition. It means that The chain is not irreducible.

(also there is an error in your matrix on the first line as the sum of probabilities should be equal to 1).

  • $\begingroup$ It means that The chain is not irreducible - do you mean to say the chain is irreducible? $\endgroup$ Apr 16 '13 at 11:31
  • $\begingroup$ irreducible is when there is only one communication class. This is not the case. So the chain is not irreducible. $\endgroup$
    – lcrmorin
    Apr 16 '13 at 12:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.