# Solving difference equation for stationary distribution of Markov Chain

Say I have the following matrix where the state space is {0,1,2,3,4}

\begin{bmatrix} 1-p & p & 0 & 0 & 0\\ 1-p & 0 & p & 0 & 0\\ 0 & 1-p & 0 & p & 0\\ 0 & 0 & 1-p & 0 & p\\ 0 & 0 & 0 & 1-p & p \end{bmatrix}

where $0 < p < 1$

I have derived a general solution for calculating the stationary distribution when $p\neq\frac{1}{2}$:

$$\pi_{i}=A\left ( \frac{p}{1-p} \right )^i + B$$

I would like to determine the values of the constants A and B should be simple enough, but I'm not sure of the boundary conditions. I know the stationary distribution should sum to 1, i.e.

$$\pi_{0}+\pi_{1}+\pi_{2}+\pi_{3}+\pi_{4}=1$$

For ease, I would like to determine the boundary condition at $\pi_{0}$ as this gives $\pi_{0}=A+B$

It seems like it should be $\pi_{0}=A+B=(1-p)$ but it doesn't look right when I try to solve it.

• By solving the eigensystem of the transition matrix, I got the following expression for the stationary distribution: $\left(\frac{(p-1)^4}{(p-1) p ((p-1) p+3)+1},-\frac{(p-1)^3 p}{(p-1) p ((p-1) p+3)+1},\frac{(p-1)^2 p^2}{(p-1) p ((p-1) p+3)+1},-\frac{(p-1) p^3}{(p-1) p ((p-1) p+3)+1},\frac{p^4}{(p-1) p ((p-1) p+3)+1}\right)$. I checked it numerically using some values for $p$ and the results match up. – COOLSerdash Mar 18 '17 at 15:00
• Sorry, I'm not entirely sure what you did there. I just want to know how to calculate A and B to obtain a general solution. I believe the boundary conditions should be $(1-\beta)$ for $\pi_{0}$ and $0$ for $\pi_{1}$ does this seem correct? – ChiPhi85 Mar 18 '17 at 17:27
• Well I already found the general solution of the stationary distribution (posted in my first comment). I don't know how to find $A$ or $B$, sorry. – COOLSerdash Mar 18 '17 at 17:51

You do not explain how you found this general solution but it might be based on the identities $$\pi_k=p\pi_{k-1}+(1-p)\pi_{k+1}$$ for every $1\leqslant k\leqslant3$. If $p\ne\frac12$, introducing the parameter $$r=\frac{p}{1-p}$$ these are indeed solved by $$\pi_k=Ar^k+B$$ for every $0\leqslant k\leqslant4$. Perfect. But... all this is forgetting the boundary identities, namely, $$\pi_0=(1-p)\pi_0+(1-p)\pi_1\qquad\pi_4=p\pi_3+p\pi_4$$ These can be rewritten as $$\pi_1=r\pi_0\qquad\pi_4=r\pi_3$$ thus, we know that $$Ar+B=r(A+B)\qquad Ar^4+B=r(Ar^3+B)$$ that is, $$B=0$$ that is, $$\pi_k=Ar^k$$ for every $0\leqslant k\leqslant4$. The normalization condition mentioned in your post allows to determine $A$ and one gets finally, for every $0\leqslant k\leqslant4$, $$\pi_k=\frac{1-r}{1-r^5}r^k$$