0
$\begingroup$

I am given that the probability to reach a specific absorbing state $s$, from states 1, ..., M as $a_1, \cdots, a_M$, which are unique solutions to equations $a_s = 1$, $a_0 = 0$ for all absorbing states such that $i \neq s$, and $a_i = \sum_i^M a_j p_{ij}$ for all transient states $i$.

Can someone show me how this summation is derived. It looks like the law of total probability, by conditioning on the next states reachable from $i$. But it looks like a different kind of law of total probability (LOTP) that what I'm used to seeing.

Usually when I see LOTP, it's something like this: $$ P(A) = \sum_k P(A, B_k) = \sum_k P(A | B_k) P(B_k). $$

For the Markov chain probability $a_i$, say $i=1, M=2$, we have

$$ a_1 = p_{11}a_1 + p_{12}a2 \\ $$

Note that $$ p_{11} = P(X_{n+1} = 1 | X_{n} = 1) \\ p_{12} = P(X_{n+1} = 2 | X_{n} = 1) \\ $$

Is this an application of LOTP?

$\endgroup$
0
$\begingroup$

A minor issue first, but one which may have caused you confusion: the lower index in your first sum should be 1, not $i$; and in the "LOTP" equation, the notation $P(A\cap B_k)$ would also be clearer than $P(A, B_k)$.

Yes, the result does use the LOTP. The key observation that allows to connect the generic LOTP equation and your problem at hand is that the index $i$ is "built into" the probability $P$ from the LOTP (so that $P$ may itself be regarded as a conditional probability). That is, if $B_k$ is the probability of going to state $k$ in one step, then $P(B_k)$ is actually the probability of going from $i$ to $k$ in one step, thus corresponding to $p_{ik}$. That the second factor in the equation that you wish to prove is just $a_j$ is due to the Markov property.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.