# Classification of states in Markov Chain

Question

Consider the following transition matrix:

 P=
0     0     1     0     0     0
0     0     0     0     0     1
0     0     0     0     1     0
1/4   1/4    0    1/2    0     0
1     0     0     0     0     0
0    1/3    0     0     0    2/3


a) Which states are transient? b) Which states are recurrent? c) Identify all closed sets of states. d) Is this chain ergodic?

Dear friends ,

I have thought the states are as follows, {1,5} {0,2,4} recurrent since they communicate with each other {3} transient

and since state 3 does not communicate with other states, it is not an ergodic Mc

Am I correct?

Thank you..

• Dear @whuber , I don't know how to write matrix.It is not mentioned in the link also :/ – StocSim Mar 16 '13 at 10:42
• I have at least roughly fixed the matrix for you, by selecting the text for the matrix and then clicking on the $\{\}$ symbol to make it monospaced text and then lining it up. – Glen_b -Reinstate Monica Mar 16 '13 at 11:22
• Stochastic processes is a subject of statistics rather than 'math' math is deterministic,stat is stochastic. thx – StocSim Mar 16 '13 at 17:32
• @StocSim I think the relevant determinant of what makes a question appropriate on CV is the scope given in the faq; that would be the basis on which to argue that it belongs here rather than math.stackexchange.com – Glen_b -Reinstate Monica Mar 16 '13 at 17:40

Let the state space of the Markov Chain be $S=\{1,2,3,4,5,6\}$. Now draw the state transition diagram. (a). From the figure, we observe that $\{4\}$, and $\{6\}$ form non-closed communicating classes. State $2$ does not communicate even with itself and such a state is called a non-return state. Hence, the states 2, 4 and 6 are transient.
(b)&(c). The class $\{1,3,5\}$ is a closed-communicating class. Hence, states 1, 3 and 5 are recurrent states.
(c). There is only one closed-communicating class, $\{1,3,5\}$.