If $X_n$ is a Markov Chain. P(0,0) = 0.5, P(0,1) = 0.5. For all states x > 0, P(x, x) = 0.5, P(x, x+1) = P(x, x-1) = 0.25. My goal is to find $P_0$ ($T_0$ < $T_5$), which is the probability of starting at state 0, getting to state 0 before state 5. I know there is a formula for $P_x$ ($T_0$ < $T_5$) when 0 < x < 5, but I have no idea how to deal with the case when x = 0. Thanks!
1 Answer
$\begingroup$
$\endgroup$
I'm a bit rusty on Markov chains, but suppose you know how to calculate $P_1(T_0 < T_5)$. Then Starting from state 0, you have two cases:
- The state remains 0 with probability $P(0, 0) = 0.5$, after which it's certain that you reach state 0 before state 5, or
- The state changes to 1 with probability $P(0, 1) = 0.5$, after which you have probability $P_1(T_0 < T_5)$ of reaching state 0 before state 5.
So $P_0(T_0 < T_5)$ would just be $P(0, 0) \cdot 1 + P(0, 1) \cdot P_1(T_0 < T_5)$.
