# How to find the equilibrium distribution of a discrete time Markov Chain

I want to find the all the equilibrium distribution of this markov chain

By letting w1= α, where 0<= α <=1, I got w2= (7/15)α w3 = (2/3)α w4= (3/5)α

But I am not sure if its correct.

Also, does this Markov chain has limiting distribution?

• can you show your calculations as well – gunes Nov 12 '20 at 12:47
• If this is homework, you should probably add the self-study tag as well. – chl Nov 12 '20 at 17:54
• Hints: Obviously ergodic because $\mathbf{P}$-matrix has all positive elements. Stationary distribution (4-vector with elements summing to 1) with $\sigma$ with $\sigma\mathbf{P} = \sigma.$ But notice that $\mathbf{P}$ is doubly stochastic (that is columns as well as rows sum to $1).$ – BruceET Nov 12 '20 at 23:12

### Eigenvectors and eigenvalues

For an equilibrium distribution, you have

$$P \cdot v = \lambda v \quad \text{where \lambda = 1}$$

So you will need to look for the eigenvector $$v$$ that relates to the eigenvalue $$1$$.

You can compute it by solving the following homogeneous system of linear equations

$$(P-\lambda) v = \begin{bmatrix} \frac{1}{2}-\lambda&\frac{1}{4}&\frac{1}{8}&\frac{1}{8} \\ \frac{1}{4}&\frac{1}{8}-\lambda&\frac{1}{8}&\frac{1}{2} \\ \frac{1}{8}&\frac{1}{8}&\frac{1}{2}-\lambda&\frac{1}{4} \\ \frac{1}{8}&\frac{1}{2}&\frac{1}{4}&\frac{1}{8}-\lambda \\ \end{bmatrix} v = 0$$

which you could do by writing the matrix in row echelon form with Gaussian elimination

### Sudoku

Maybe you are supposed to solve it like above, but note how the values are distributed and see how the matrix looks like a sudoku or a magic square (all rows and columns sum up to 1).

Can you use this knowledge to easily think of an eigenvector with eigenvalue 1?

### Solving with computer code

You can solve the above problem automatically and fast with a computer.

m <- rbind(c(4,2,1,1),
c(2,1,1,4),
c(1,1,4,2),
c(1,4,2,1))/8
eigen(m)


which gives


$values 3 1.0000000 0.3952847 0.2500000 -0.3952847 $vectors
[,1]       [,2] [,3]       [,4]
[1,] -0.5  0.6979762 -0.5 -0.1132660
[2,] -0.5  0.1132660  0.5  0.6979762
[3,] -0.5 -0.6979762 -0.5  0.1132660
[4,] -0.5 -0.1132660  0.5 -0.6979762


### Limiting distribution

Does this Markov chain has limiting distribution?

The starting vector can be written as a sum of the eigenvectors

$$x_0 = a_1 v_1 + a_2 v_2 + a_3 v_3 + a_4 v_4$$

Then you will have:

$$\begin{array}{} x_1 &=& P \cdot x_0\\ &=& P \cdot (a_1 v_1 + a_2 v_2 + a_3 v_3 + a_4 v_4) \\&=& a_1 \lambda_1 v_1 + \lambda_2 a_2 v_2 + \lambda_3 a_3 v_3 + \lambda_4 a_4 v_4 \end{array}$$

and you can continue this reasoning to

$$x_n = \lambda_1^n a_1 v_1 + \lambda_2^n a_2 v_2 + \lambda_3^n a_3 v_3 + \lambda_4^n a_4 v_4$$

Now consider the limiting behavior for the values of $$\lambda_i^n$$ and argue which value $$x_n$$ will converge to for $$n \to \infty$$

• equilibrium distribution is the probability vector so all the values should be positive and they all should sum up to 1, but in your solutions there are negative values. – bluelagoon Nov 12 '20 at 15:25
• @bluelagoon you can rescale eigenvectors however you like. The R function scales them such that the length (mean square of the components) equals 1. If $P\cdot v = v$ then also $P \cdot (cv) = (cv)$ – Sextus Empiricus Nov 12 '20 at 15:26