# writing down markov chain transition matrix

Question:

An experimental animal can stay in room-A until 1 minute,and it can stay in room-B until 2 minutes. There exist deadly gases in room-C. One room among these three rooms is being randomly chosen per a minute. And The animal goes to this chosen room.

How to write down transition matrix?

State space is

S ={ 1minute in room-A, 1minute in room-B, 2minutes in room-B, room-C}={0, 1, 2, 3} (respevtively)

Transition matrix has been written like that;

$$\mathcal P = \begin{bmatrix} 1/3 & 1/3 & 0 & 1/3 \\ 0 & 0 & 1 & 0 \\ 1/3 & 1/3 & 0 & 1/3 \\ 0 & 0 & 0 & 1 \end{bmatrix}$$

But, I cannot understand how to write down this transition matrix?

Why $P_{00}^{(1)}=1/3$ or $P_{12}^{(1)}=1$ or ...? Please how to write these probability values in the matrix? Explain it with reasons. thank you.

• Could you please clarify your question? You ask, "How to write down transition matrix?", yet also include a transition matrix in the statement of your question. Are you saying that the reported matrix is correct, but you don't understand why? Or that you think this matrix is incorrect? Or what? – Arthur Small Mar 30 '15 at 17:37
• Also: is this a homework question? If so, please tag it as [self-study]. – Arthur Small Mar 30 '15 at 17:37
• no no this matrix is perfectly correct.i wrote both question and its answer. i did this example in class. but i dont understand how to determine probability values in this matrix.i have asked this. hopefully i can explain what i dont understand. @ArthurSmall – B11b Mar 30 '15 at 17:40
• accourding to given question, why do you write the matrix? @ArthurSmall – B11b Mar 30 '15 at 17:47

If I understand your question correctly...

There are four possible states:

• 0: In Room A
• 1: In Room B, first of two minutes
• 2: In Room B, second of two minutes
• 3: In Room C

The system changes state each minute. Consider each of the four starting positions:

State 0: If the animal is currently in Room A, then it either stays in Room A, or moves to Room B, or moves to Room C, each with probability 1/3. If the animal moves to Room B, then it starts the first minute of a 2-minute stay. Hence the shift in state is from State 0 to State 1. There is no possibility to shift directly from State 0 to State 2: the animal can't shift from not being in Room B, directly into the second of two minutes in Room B. Hence P_{02} = 0.

State 1: If the animal is currently in Room B and in the first minute of a two-minute stay, then the animal is in State 1. In this case, the animal must remain in Room B for an additional minute. In so doing, the animal shifts from State 1 to State 2. Hence P_{12}=1.

State 2: If the animal is currently in Room B and in the second minute of a two-minute stay, then the animal either shifts to Room A (State 0), starts a new 2-minute stay in Room B (goes back to State 1), or shifts to Room C (State 3), each with Prob = 1/3.

State 3 is an absorbing state: P_{33}=1.