Equilibrium distribution of Markov chain

Question

The transition matrix is $$P =\begin{bmatrix} \frac12 & \frac12 & 0 & 0 \\ \frac12 & \frac12 & 0 & 0 \\ 0 & 0 & \frac13 & \frac23 \\ 0 & 0 & \frac13 & \frac23\end{bmatrix}$$

Now the question is how can I find all equilibrium distribution of this chain. I have got $2$ which are $(0.5, 0.5, 0, 0)$ and $(0, 0, \frac13, \frac23)$

In order to find the other equilibrium distribution I tried $wP = w$ and I got $5$ equation upon which solving them I got $w_1=w_2, w_2=w_2, w_4 = 2w_3$ and $w_3=w_3$, so I conclude that any probability vector of form $(a , a, b, 1-2a-b)(a,b \in \mathbb{R})$ can be equilibrium distribution but when I tried with few numbers for $a$ and $b$ it not giving me the same probability vector that I started with.

I can't see what I am doing wrong

Answer 1

Due to the sum to one criterion and $w_4=2w_3,$ the $a$ and $b$ also has to be chosen to satisfy this condition:

$$1-2a-b=2b$$

I think you miss out this condition, that is knowing $a$ would completely determine $b$. Also, we need each component to be nonnegative.

To recover your first solution, you can let $a=\frac12, b=0$.

To get your second solution, you can let $a=0,b=\frac13 $.

The general solution is the convex hull of the solution for each of the classes.

$$\alpha \left(\frac12, \frac12, 0, 0\right) + (1-\alpha)\left( 0,0, \frac13, \frac23\right)$$

where $0 \le \alpha \le 1$.

Answer 2

This Markov Chain is not irreducible and is therefore not ergodic. That is the reason why there is no unique equilibrium distribution. More specifically: nonergodicity entails that the equilibrium distribution depends on the distribution of the initial state $X_0$ of this chain. This chain has two irreducible classes, {1,2} consisting of states 1 and 2, and {3,4} consisting of states 3 and 4. In the answer of @Siong Thye Goh, the parameter $\alpha$ has a precise interpretation, namely: $$ \alpha = \text{Prob}[X_0 \in \{1,2\}] $$

Answer 3

We have 3 restricting conditions on $w_{1}$, $w_{2}$, $w_{3}$ and $w_{4}$: (1) $w_{1}=w_{2}$, (2) $w_{4}=2w_{3}$, and (3) $w_{1}+w_{2}+w_{3}+w_{4}=1$. The general solution of (1) is $[w_{1},w_{2}] = w_{1}\times[1,1]$. The general solution of (2) is $[w_{3},w_{4}] = w_{3}\times[1,2]$. Finally from (3), we have $w_{1}+w_{2}+w_{3}+w_{4}=2w_{1} + 3w_{3}=1$. This last equation,

$2w_{1} + 3w_{3}=1$,

gives the general solution. To satisfy this equation, obviously, $w_{1}\leq1/2$ and $w_{3}\leq1/3$. For example, $w_{1}=1/4$ and $w_{3}=1/6$.

Therefore, $w_{1}$ and $w_{3}$ can be expressed as $w_{1}=1/2\times \alpha$ and $w_{3}=1/3 \times \beta$ respectively, where $0\leq\alpha\leq1$ and $0\leq\beta \leq1$. Substituting to above equation, we get $\beta = 1-\alpha$.