The exercise is asking for the stationary distribution, the estimated time to get from state $0$ to state $4,$ and to conclude if the chain is time-reversible.

So I have the following transition matrix:

$$P=\left[\begin{matrix} 1/2&1/2&0&0&0&\cdots\\ 1/2&0&1/2&0&0&\cdots\\ 1/2&0&0&1/2&0&\cdots\\ 1/2&0&0&0&1/2&\cdots\\ 1/2&0&0&0&0&\cdots\\ \vdots&\vdots&\vdots&\vdots&\vdots&\ddots \end{matrix}\right]$$

This means $P_{i,0} =1/2,$ $P_{i,i+1} = 1/2,$ and $P_{ij}=0$ in every other case.

I have progressed up to this point:

\begin{align*} n_1&=\frac12\,n_1+\frac12\,n_2+\frac12\,n_3+\dots\\ n_2&=\frac12\,n_1\\ n_3&=\frac12\,n_2\\ &\vdots\\ n_k&=\frac12\,n_{k-1}\\ \sum_{k=1}^\infty n_k&=1. \end{align*}

I'm not sure how to solve the problem, though.

Any help would be very much appreciated.


1 Answer 1


We can rewrite manipulate the first equation by noticing that $\pi_1+\pi_2+\pi_3+\cdots=1$. This gives us the following:

$$\begin{align} \pi_1&=\frac12\pi_2+\frac12\pi_3+\frac12\pi_4+\cdots\\ &=\frac12\left(\pi_2+\pi_3+\pi_4+\cdots\right)\\ &=\frac12(1-\pi_1) \end{align}$$

Solving $\pi_1=\frac12(1-\pi_1)$ gives us $\pi_1=\frac13$. Then, each term is half of the previous one ($\pi_n=\frac12\pi_{n-1}$), so we have $\pi_n=\frac13\left(\frac12\right)^{n-1}$.

Hope it was helpful!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.