Equilibrium distribution of Markov chain

The transition matrix is $$P =\begin{bmatrix} \frac12 & \frac12 & 0 & 0 \\ \frac12 & \frac12 & 0 & 0 \\ 0 & 0 & \frac13 & \frac23 \\ 0 & 0 & \frac13 & \frac23\end{bmatrix}$$

Now the question is how can I find all equilibrium distribution of this chain. I have got $$2$$ which are $$(0.5, 0.5, 0, 0)$$ and $$(0, 0, \frac13, \frac23)$$

In order to find the other equilibrium distribution I tried $$wP = w$$ and I got $$5$$ equation upon which solving them I got $$w_1=w_2, w_2=w_2, w_4 = 2w_3$$ and $$w_3=w_3$$, so I conclude that any probability vector of form $$(a , a, b, 1-2a-b)(a,b \in \mathbb{R})$$ can be equilibrium distribution but when I tried with few numbers for $$a$$ and $$b$$ it not giving me the same probability vector that I started with.

I can't see what I am doing wrong

3 Answers

Due to the sum to one criterion and $$w_4=2w_3,$$ the $$a$$ and $$b$$ also has to be chosen to satisfy this condition:

$$1-2a-b=2b$$

I think you miss out this condition, that is knowing $$a$$ would completely determine $$b$$. Also, we need each component to be nonnegative.

To recover your first solution, you can let $$a=\frac12, b=0$$.

To get your second solution, you can let $$a=0,b=\frac13$$.

The general solution is the convex hull of the solution for each of the classes.

$$\alpha \left(\frac12, \frac12, 0, 0\right) + (1-\alpha)\left( 0,0, \frac13, \frac23\right)$$

where $$0 \le \alpha \le 1$$.

• Can you please explain how you got the final convex solution? – bluelagoon Oct 22 '20 at 15:29
• I know the theoretical result before hand. But if you wish to derive it, first, express $b$ in terms of $a$ and then you study the range of $a$ to make each component nonnegative. – Siong Thye Goh Oct 22 '20 at 15:36
• I tried doing that and I got (a a (1-2a)/3 (2-4a)/3 ) where 0 ≤ a ≤ 1/2, is it correct? – bluelagoon Oct 22 '20 at 22:51
• Let $a=\frac12 \theta$ and our solution is the same. Just use convex combination in the future, it's much faster :). – Siong Thye Goh Oct 23 '20 at 1:49
• @user295357 your solution can be expressed in the form given in the answer with $\alpha=\frac{2}{5}$, $\frac{2}{5}(1/2,1/2,0,0)+\frac{3}{5}(0,0,1/3,2/3)$ – Tyberius Oct 23 '20 at 18:06

This Markov Chain is not irreducible and is therefore not ergodic. That is the reason why there is no unique equilibrium distribution. More specifically: nonergodicity entails that the equilibrium distribution depends on the distribution of the initial state $$X_0$$ of this chain. This chain has two irreducible classes, {1,2} consisting of states 1 and 2, and {3,4} consisting of states 3 and 4. In the answer of @Siong Thye Goh, the parameter $$\alpha$$ has a precise interpretation, namely: $$\alpha = \text{Prob}[X_0 \in \{1,2\}]$$

• I don't think that this answers the question. OP already knows the invariant distribution is not unique. – Federico Poloni Oct 23 '20 at 11:12

We have 3 restricting conditions on $$w_{1}$$, $$w_{2}$$, $$w_{3}$$ and $$w_{4}$$: (1) $$w_{1}=w_{2}$$, (2) $$w_{4}=2w_{3}$$, and (3) $$w_{1}+w_{2}+w_{3}+w_{4}=1$$. The general solution of (1) is $$[w_{1},w_{2}] = w_{1}\times[1,1]$$. The general solution of (2) is $$[w_{3},w_{4}] = w_{3}\times[1,2]$$. Finally from (3), we have $$w_{1}+w_{2}+w_{3}+w_{4}=2w_{1} + 3w_{3}=1$$. This last equation,

$$2w_{1} + 3w_{3}=1$$,

gives the general solution. To satisfy this equation, obviously, $$w_{1}\leq1/2$$ and $$w_{3}\leq1/3$$. For example, $$w_{1}=1/4$$ and $$w_{3}=1/6$$.

Therefore, $$w_{1}$$ and $$w_{3}$$ can be expressed as $$w_{1}=1/2\times \alpha$$ and $$w_{3}=1/3 \times \beta$$ respectively, where $$0\leq\alpha\leq1$$ and $$0\leq\beta \leq1$$. Substituting to above equation, we get $$\beta = 1-\alpha$$.