3
$\begingroup$

Say I have the following matrix where the state space is {0,1,2,3,4}

\begin{bmatrix} 1-p & p & 0 & 0 & 0\\ 1-p & 0 & p & 0 & 0\\ 0 & 1-p & 0 & p & 0\\ 0 & 0 & 1-p & 0 & p\\ 0 & 0 & 0 & 1-p & p \end{bmatrix}


where $ 0 < p < 1 $


I have derived a general solution for calculating the stationary distribution when $p\neq\frac{1}{2}$:


$$ \pi_{i}=A\left ( \frac{p}{1-p} \right )^i + B $$


I would like to determine the values of the constants A and B should be simple enough, but I'm not sure of the boundary conditions. I know the stationary distribution should sum to 1, i.e.


$$ \pi_{0}+\pi_{1}+\pi_{2}+\pi_{3}+\pi_{4}=1 $$


For ease, I would like to determine the boundary condition at $\pi_{0}$ as this gives $\pi_{0}=A+B$


It seems like it should be $\pi_{0}=A+B=(1-p)$ but it doesn't look right when I try to solve it.

$\endgroup$
  • $\begingroup$ By solving the eigensystem of the transition matrix, I got the following expression for the stationary distribution: $\left(\frac{(p-1)^4}{(p-1) p ((p-1) p+3)+1},-\frac{(p-1)^3 p}{(p-1) p ((p-1) p+3)+1},\frac{(p-1)^2 p^2}{(p-1) p ((p-1) p+3)+1},-\frac{(p-1) p^3}{(p-1) p ((p-1) p+3)+1},\frac{p^4}{(p-1) p ((p-1) p+3)+1}\right)$. I checked it numerically using some values for $p$ and the results match up. $\endgroup$ – COOLSerdash Mar 18 '17 at 15:00
  • $\begingroup$ Sorry, I'm not entirely sure what you did there. I just want to know how to calculate A and B to obtain a general solution. I believe the boundary conditions should be $(1-\beta)$ for $\pi_{0}$ and $0$ for $\pi_{1}$ does this seem correct? $\endgroup$ – ChiPhi85 Mar 18 '17 at 17:27
  • $\begingroup$ Well I already found the general solution of the stationary distribution (posted in my first comment). I don't know how to find $A$ or $B$, sorry. $\endgroup$ – COOLSerdash Mar 18 '17 at 17:51
1
$\begingroup$

You do not explain how you found this general solution but it might be based on the identities $$\pi_k=p\pi_{k-1}+(1-p)\pi_{k+1}$$ for every $1\leqslant k\leqslant3$. If $p\ne\frac12$, introducing the parameter $$r=\frac{p}{1-p}$$ these are indeed solved by $$\pi_k=Ar^k+B$$ for every $0\leqslant k\leqslant4$. Perfect. But... all this is forgetting the boundary identities, namely, $$\pi_0=(1-p)\pi_0+(1-p)\pi_1\qquad\pi_4=p\pi_3+p\pi_4$$ These can be rewritten as $$\pi_1=r\pi_0\qquad\pi_4=r\pi_3$$ thus, we know that $$Ar+B=r(A+B)\qquad Ar^4+B=r(Ar^3+B)$$ that is, $$B=0$$ that is, $$\pi_k=Ar^k$$ for every $0\leqslant k\leqslant4$. The normalization condition mentioned in your post allows to determine $A$ and one gets finally, for every $0\leqslant k\leqslant4$, $$\pi_k=\frac{1-r}{1-r^5}r^k$$

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.