The transition matrix is $$P =\begin{bmatrix} \frac12 & \frac12 & 0 & 0 \\ \frac12 & \frac12 & 0 & 0 \\ 0 & 0 & \frac13 & \frac23 \\ 0 & 0 & \frac13 & \frac23\end{bmatrix}$$

Now the question is how can I find all equilibrium distribution of this chain. I have got $2$ which are $(0.5, 0.5, 0, 0)$ and $(0, 0, \frac13, \frac23)$

In order to find the other equilibrium distribution I tried $wP = w$ and I got $5$ equation upon which solving them I got $w_1=w_2, w_2=w_2, w_4 = 2w_3$ and $w_3=w_3$, so I conclude that any probability vector of form $(a , a, b, 1-2a-b)(a,b \in \mathbb{R})$ can be equilibrium distribution but when I tried with few numbers for $a$ and $b$ it not giving me the same probability vector that I started with.

I can't see what I am doing wrong


Due to the sum to one criterion and $w_4=2w_3,$ the $a$ and $b$ also has to be chosen to satisfy this condition:


I think you miss out this condition, that is knowing $a$ would completely determine $b$. Also, we need each component to be nonnegative.

To recover your first solution, you can let $a=\frac12, b=0$.

To get your second solution, you can let $a=0,b=\frac13 $.

The general solution is the convex hull of the solution for each of the classes.

$$\alpha \left(\frac12, \frac12, 0, 0\right) + (1-\alpha)\left( 0,0, \frac13, \frac23\right)$$

where $0 \le \alpha \le 1$.

  • $\begingroup$ Can you please explain how you got the final convex solution? $\endgroup$ – bluelagoon Oct 22 '20 at 15:29
  • $\begingroup$ I know the theoretical result before hand. But if you wish to derive it, first, express $b$ in terms of $a$ and then you study the range of $a$ to make each component nonnegative. $\endgroup$ – Siong Thye Goh Oct 22 '20 at 15:36
  • $\begingroup$ I tried doing that and I got (a a (1-2a)/3 (2-4a)/3 ) where 0 ≤ a ≤ 1/2, is it correct? $\endgroup$ – bluelagoon Oct 22 '20 at 22:51
  • 1
    $\begingroup$ Let $a=\frac12 \theta$ and our solution is the same. Just use convex combination in the future, it's much faster :). $\endgroup$ – Siong Thye Goh Oct 23 '20 at 1:49
  • $\begingroup$ @user295357 your solution can be expressed in the form given in the answer with $\alpha=\frac{2}{5}$, $\frac{2}{5}(1/2,1/2,0,0)+\frac{3}{5}(0,0,1/3,2/3)$ $\endgroup$ – Tyberius Oct 23 '20 at 18:06

This Markov Chain is not irreducible and is therefore not ergodic. That is the reason why there is no unique equilibrium distribution. More specifically: nonergodicity entails that the equilibrium distribution depends on the distribution of the initial state $X_0$ of this chain. This chain has two irreducible classes, {1,2} consisting of states 1 and 2, and {3,4} consisting of states 3 and 4. In the answer of @Siong Thye Goh, the parameter $\alpha$ has a precise interpretation, namely: $$ \alpha = \text{Prob}[X_0 \in \{1,2\}] $$

  • 2
    $\begingroup$ I don't think that this answers the question. OP already knows the invariant distribution is not unique. $\endgroup$ – Federico Poloni Oct 23 '20 at 11:12

We have 3 restricting conditions on $w_{1}$, $w_{2}$, $w_{3}$ and $w_{4}$: (1) $w_{1}=w_{2}$, (2) $w_{4}=2w_{3}$, and (3) $w_{1}+w_{2}+w_{3}+w_{4}=1$. The general solution of (1) is $[w_{1},w_{2}] = w_{1}\times[1,1]$. The general solution of (2) is $[w_{3},w_{4}] = w_{3}\times[1,2]$. Finally from (3), we have $w_{1}+w_{2}+w_{3}+w_{4}=2w_{1} + 3w_{3}=1$. This last equation,

$2w_{1} + 3w_{3}=1$,

gives the general solution. To satisfy this equation, obviously, $w_{1}\leq1/2$ and $w_{3}\leq1/3$. For example, $w_{1}=1/4$ and $w_{3}=1/6$.

Therefore, $w_{1}$ and $w_{3}$ can be expressed as $w_{1}=1/2\times \alpha$ and $w_{3}=1/3 \times \beta$ respectively, where $0\leq\alpha\leq1$ and $0\leq\beta \leq1$. Substituting to above equation, we get $\beta = 1-\alpha$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.