The transition matrix is $$P =\begin{bmatrix} \frac12 & \frac12 & 0 & 0 \\ \frac12 & \frac12 & 0 & 0 \\ 0 & 0 & \frac13 & \frac23 \\ 0 & 0 & \frac13 & \frac23\end{bmatrix}$$

Now the question is how can I find all equilibrium distribution of this chain. I have got $2$ which are $(0.5, 0.5, 0, 0)$ and $(0, 0, \frac13, \frac23)$

In order to find the other equilibrium distribution I tried $wP = w$ and I got $5$ equation upon which solving them I got $w_1=w_2, w_2=w_2, w_4 = 2w_3$ and $w_3=w_3$, so I conclude that any probability vector of form $(a , a, b, 1-2a-b)(a,b \in \mathbb{R})$ can be equilibrium distribution but when I tried with few numbers for $a$ and $b$ it not giving me the same probability vector that I started with.

I can't see what I am doing wrong


3 Answers 3


Due to the sum to one criterion and $w_4=2w_3,$ the $a$ and $b$ also has to be chosen to satisfy this condition:


I think you miss out this condition, that is knowing $a$ would completely determine $b$. Also, we need each component to be nonnegative.

To recover your first solution, you can let $a=\frac12, b=0$.

To get your second solution, you can let $a=0,b=\frac13 $.

The general solution is the convex hull of the solution for each of the classes.

$$\alpha \left(\frac12, \frac12, 0, 0\right) + (1-\alpha)\left( 0,0, \frac13, \frac23\right)$$

where $0 \le \alpha \le 1$.

  • $\begingroup$ Can you please explain how you got the final convex solution? $\endgroup$
    – bluelagoon
    Commented Oct 22, 2020 at 15:29
  • $\begingroup$ I know the theoretical result before hand. But if you wish to derive it, first, express $b$ in terms of $a$ and then you study the range of $a$ to make each component nonnegative. $\endgroup$ Commented Oct 22, 2020 at 15:36
  • $\begingroup$ I tried doing that and I got (a a (1-2a)/3 (2-4a)/3 ) where 0 ≤ a ≤ 1/2, is it correct? $\endgroup$
    – bluelagoon
    Commented Oct 22, 2020 at 22:51
  • 1
    $\begingroup$ Let $a=\frac12 \theta$ and our solution is the same. Just use convex combination in the future, it's much faster :). $\endgroup$ Commented Oct 23, 2020 at 1:49
  • $\begingroup$ @user295357 your solution can be expressed in the form given in the answer with $\alpha=\frac{2}{5}$, $\frac{2}{5}(1/2,1/2,0,0)+\frac{3}{5}(0,0,1/3,2/3)$ $\endgroup$
    – Tyberius
    Commented Oct 23, 2020 at 18:06

This Markov Chain is not irreducible and is therefore not ergodic. That is the reason why there is no unique equilibrium distribution. More specifically: nonergodicity entails that the equilibrium distribution depends on the distribution of the initial state $X_0$ of this chain. This chain has two irreducible classes, {1,2} consisting of states 1 and 2, and {3,4} consisting of states 3 and 4. In the answer of @Siong Thye Goh, the parameter $\alpha$ has a precise interpretation, namely: $$ \alpha = \text{Prob}[X_0 \in \{1,2\}] $$

  • 2
    $\begingroup$ I don't think that this answers the question. OP already knows the invariant distribution is not unique. $\endgroup$ Commented Oct 23, 2020 at 11:12

We have 3 restricting conditions on $w_{1}$, $w_{2}$, $w_{3}$ and $w_{4}$: (1) $w_{1}=w_{2}$, (2) $w_{4}=2w_{3}$, and (3) $w_{1}+w_{2}+w_{3}+w_{4}=1$. The general solution of (1) is $[w_{1},w_{2}] = w_{1}\times[1,1]$. The general solution of (2) is $[w_{3},w_{4}] = w_{3}\times[1,2]$. Finally from (3), we have $w_{1}+w_{2}+w_{3}+w_{4}=2w_{1} + 3w_{3}=1$. This last equation,

$2w_{1} + 3w_{3}=1$,

gives the general solution. To satisfy this equation, obviously, $w_{1}\leq1/2$ and $w_{3}\leq1/3$. For example, $w_{1}=1/4$ and $w_{3}=1/6$.

Therefore, $w_{1}$ and $w_{3}$ can be expressed as $w_{1}=1/2\times \alpha$ and $w_{3}=1/3 \times \beta$ respectively, where $0\leq\alpha\leq1$ and $0\leq\beta \leq1$. Substituting to above equation, we get $\beta = 1-\alpha$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.