0
$\begingroup$

If $X_n$ is a Markov Chain. P(0,0) = 0.5, P(0,1) = 0.5. For all states x > 0, P(x, x) = 0.5, P(x, x+1) = P(x, x-1) = 0.25. My goal is to find $P_0$ ($T_0$ < $T_5$), which is the probability of starting at state 0, getting to state 0 before state 5. I know there is a formula for $P_x$ ($T_0$ < $T_5$) when 0 < x < 5, but I have no idea how to deal with the case when x = 0. Thanks!

$\endgroup$
1
$\begingroup$

I'm a bit rusty on Markov chains, but suppose you know how to calculate $P_1(T_0 < T_5)$. Then Starting from state 0, you have two cases:

  • The state remains 0 with probability $P(0, 0) = 0.5$, after which it's certain that you reach state 0 before state 5, or
  • The state changes to 1 with probability $P(0, 1) = 0.5$, after which you have probability $P_1(T_0 < T_5)$ of reaching state 0 before state 5.

So $P_0(T_0 < T_5)$ would just be $P(0, 0) \cdot 1 + P(0, 1) \cdot P_1(T_0 < T_5)$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.