writing down markov chain transition matrix Question:
An experimental animal can stay in room-A until 1 minute,and it can stay in room-B until 2 minutes. There exist deadly gases in room-C. One room among these three rooms is being randomly chosen per a minute. And The animal goes to this chosen room.
How to write down transition matrix?

Answer
State space is
S ={ 1minute in room-A, 1minute in room-B, 2minutes in room-B, room-C}={0, 1, 2, 3} (respevtively)
Transition matrix has been written like that;
$$\mathcal P = \begin{bmatrix} 1/3 & 1/3 & 0 & 1/3 \\ 0 & 0 & 1 & 0  \\ 1/3 & 1/3 & 0 & 1/3  \\ 0 & 0 & 0 & 1  \end{bmatrix}$$ 

But, I cannot understand how to write down this transition matrix? 
Why $P_{00}^{(1)}=1/3$ or $P_{12}^{(1)}=1$ or ...? Please how to write these probability values in the matrix? Explain it with reasons. thank you.
 A: If I understand your question correctly...
There are four possible states:


*

*0: In Room A

*1: In Room B, first of two minutes

*2: In Room B, second of two minutes

*3: In Room C


The system changes state each minute. Consider each of the four starting positions:
State 0: If the animal is currently in Room A, then it either stays in Room A, or moves to Room B, or moves to Room C, each with probability 1/3. If the animal moves to Room B, then it starts the first minute of a 2-minute stay. Hence the shift in state is from State 0 to State 1. There is no possibility to shift directly from State 0 to State 2: the animal can't shift from not being in Room B, directly into the second of two minutes in Room B. Hence P_{02} = 0. 
State 1: If the animal is currently in Room B and in the first minute of a two-minute stay, then the animal is in State 1. In this case, the animal must remain in Room B for an additional minute. In so doing, the animal shifts from State 1 to State 2. Hence P_{12}=1.
State 2: If the animal is currently in Room B and in the second minute of a two-minute stay, then the animal either shifts to Room A (State 0), starts a new 2-minute stay in Room B (goes back to State 1), or shifts to Room C (State 3), each with Prob = 1/3.
State 3 is an absorbing state: P_{33}=1.
