Example of Conditional Expectation in Markov Chain

Question

My question is from the book Introduction to Probability Models, 10th edition, by Sheldon Ross.

Here is a example in the book. Consider a Markov chain with states $0, 1,\cdots , n$ having $P_{0,1} = 1, P_{i,i+1} = p, P_{i,i−1} = q = 1 − p, 1 \leq i < n$.

and suppose that we are interested in studying the time that it takes for the chain to go from state $0$ to state $n$.

One of the approaches the author uses in the book is to construct a Markov Chain. The method is as following:

let $N_i$ denote the number of additional transitions that it takes the chain when it first enters state $i$ until it enters state $i + 1$. By the Markovian property, it follows that these random variables $N_i, i = 0, . . . , n − 1$ are independent. Also, we can express $N_{0,n}$, the number of transitions that it takes the chain to go from state $0$ to state $n$, as $$ N_{0,n} = \sum_{i=0}^{n-1}N_i$$ Letting $\mu_i = E[N_i]$ we obtain, upon conditioning on the next transition after the chain enters state $i$, that for $i = 1, . . . , n − 1$

$$\mu_i = 1 + E[\text{number of additional transitions to reach } i + 1|\text{chain to } i − 1]q$$.

My question is why the author uses 1 instead of $p$ in the last equation.

To me, it sounds more reasonable that $$\begin{align} \mu_i &= E[\text{number of additional transitions to reach } i + 1 \text{ from } i|\text{chain to } i + 1]p \\& \quad+ E[\text{number of additional transitions to reach } i + 1\text{ from } i|\text{chain to } i − 1]q \\ &= p + E[\text{number of additional transitions to reach } i + 1|\text{chain to } i − 1]q \end{align}$$

Answer 1

The full problem statement can be found from Google books at Section 4.5.3 of Sheldon M. Ross "Introduction to Probability Models".

There is always at least one additional transition to get from state $i$ to state $i+1$. That is the "1" which you think should be a $p$. In reality, there is also a "missing" term, which does not appear, because it comes out to $0$; that is the term in which the $p$ actually appears.

So the "unabridged" version of the equation in the book is:

$$\mu_i = 1 + E[\text{number of additional transitions to reach } i+1|\text{chain to }i+1] p + E[\text{number of additional transitions to reach }i+1|\text{chain to }i-1] q$$

Then note that $E[\text{number of additional transitions to reach } i+1|\text{chain to }i+1] = 0$

It may help your interpretation to think of $E[\text{number of additional transitions to reach } i+1|\text{chain to }i+1]$ as meaning $E[\text{number of additional transitions to reach } i+1|\text{chain IS IN }i+1]$, which you can clearly see as being $0$.

Note that instead of writing the equation as I did, it could be written with the $1$ instead included in each of the other two terms, which perhaps you might find to be more natural, as follows:

$$\mu_i = (1 + E[\text{number of additional transitions to reach } i+1|\text{chain to }i+1]) p + (1 + E[\text{number of additional transitions to reach }i+1|\text{chain to }i-1]) q$$ and of course this reduces to the previous equation given that $p + q = 1$.