If given the Markov Chain with state space ${0,1,2,3,4,5,6}$ and transition probabilities $p(0,0) = 0.75$, $p(0,1) = 0.25$, $p(1,0) = 0.5$, $p(1,1) = 0.25$, $p(1,2) = 0.25$, $p(6,0) = 0.25$, $p(6,5) = 0.25$, $p(6,6) = 0.5$.

Edit: $p(j,0) = p(j,j-1) = p(j,j) = p(j,j+1) = 0.25$

Q. Assume the chain starts in state 1. Determine the probability it reaches state 6 before reaching state 0?

My doubt with the answer for this question: In the answer for this question, apparently we can assume that State 0 and State 6 are absorbing states. I don't understand why this is the case, or how we can just assume that?

Also, I can't understand how the Matrices associated with $Q$ and $r_2$ have been formulated in the equation $(I-Q)^{-1}r_2 = 0.0069$ (Figured this bit out.)

I don't understand how to solve the following question either:

Q. Suppose the chain starts in state 3. What is the expected number of steps until the chain is in state 3 again?

  • 1
    $\begingroup$ Either this question cannot be answered or you have not correctly transmitted it. For it to be answerable, you need to supply the full matrix, not just three out of the seven rows. I suspect it might have been intended that rows 2-5 follow some kind of pattern of $1/4$ and $1/2$ values, but that pattern is unclear. However, obviously neither $0$ nor $6$ are absorbing states: you explicitly provide nonzero probabilities to transition out of them! The last part of your question needs editing to explain what "$Q$" and "$r_2$" mean. $\endgroup$ – whuber Sep 29 '14 at 20:40
  • $\begingroup$ @whuber I've made the required changes. I forgot to add all the information. $\endgroup$ – user131983 Sep 30 '14 at 12:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.