I am bit confused after solving the following Markov Chain exercise from a textbook

Suppose that a student will be either on time or late for a particular class, and that the events that he is on time or late for the class on successive days form a markov Chain with stationary transition probabilities. Suppose also that if he is late on a given day, then the probability that he will be on time the next day is 0.8. Furthermore, if he is on time on a given day, then the probability that he will be late the next day is 0.5

If the student is late on the first day of class, what is the probability that he will be on time on the fourth day of class?

I solved the exercise by defining the following transition matrix $P$, where $L$ represents the event that the student is late and $T$ represents the event that the student arrives on time:

$$ \begin{array}{lrr} & \mbox{T} & \mbox{L} \\ \mbox{L} & 0.8 & 0.2 \\ \mbox{T} & 0.5 & 0.5\end{array} $$

Then, I can answer just by calculating $P^3$:

$$ \begin{array}{lrr} & \mbox{T} & \mbox{L} \\ \mbox{L} & 0.772 & 0.278 \\ \mbox{T} & 0.695 & 0.0305\end{array} $$

Therefore, the answer is 0.722.

However, I have realized that if I arrange the elements of the transition matrix $P$ in a different way:

$$ \begin{array}{lrr} & \mbox{L} & \mbox{T} \\ \mbox{L} & 0.2 & 0.8 \\ \mbox{T} & 0.5 & 0.5\end{array} $$

then $P^3$ is different to what I obtained before, thus changing the probability that the student is on time on the fourth day of class after being late on the first day to 0.632. This second transition matrix is the one that can be found in the solutions manual of the textbook.

Assuming that the probability shouldn't change based on the arrangement of the elements in the transition matrix, what is wrong with my first version of P?


The problem is with your first matrix: when you arrange your values like that, $P^3$ does not represent the probabilities you are looking for. To see that, consider $P^2$ and calculate the first (top left) value of the product: it will be equal to $$P(L|T)P(L|T) + P(L|L)P(T|T),$$ which is not what you want.

When you arrange your values correctly, like in the second matrix, everything works like it should. The top left value of $P^2$, for example, is equal to

$$P(L|L)P(L|L) + P(L|T)P(T|L),$$ which is equal to probability of being late on third day, given that you were late on the first.

  • $\begingroup$ Thank you. That makes sense. is there any rule of thumb to arrange the values of the matrix? $\endgroup$ – Pablo Suau Nov 14 '14 at 13:46
  • $\begingroup$ Not the rule of thumb, but a definition: if you have states $1, ..., n$, the element $p_{ij}$ of transition matrix $P$ is equal to probability to go from state $i$ to $j$. In particular, the probabilities on the diagonal are $p_{ii}$ (from state to itself), which implies that the order of states is the same in rows (looking down) and in columns (looking right). $\endgroup$ – psarka Nov 14 '14 at 14:36
  • $\begingroup$ Thank you. It's much clearer now. I just thought that the ordering was not important. $\endgroup$ – Pablo Suau Nov 14 '14 at 16:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.