# How to find the equilibrium distribution of a discrete time Markov Chain

I want to find the all the equilibrium distribution of this markov chain

By letting w1= α, where 0<= α <=1, I got w2= (7/15)α w3 = (2/3)α w4= (3/5)α

But I am not sure if its correct.

Also, does this Markov chain has limiting distribution?

• can you show your calculations as well Commented Nov 12, 2020 at 12:47
• If this is homework, you should probably add the self-study tag as well.
– chl
Commented Nov 12, 2020 at 17:54
• Hints: Obviously ergodic because $\mathbf{P}$-matrix has all positive elements. Stationary distribution (4-vector with elements summing to 1) with $\sigma$ with $\sigma\mathbf{P} = \sigma.$ But notice that $\mathbf{P}$ is doubly stochastic (that is columns as well as rows sum to $1).$ Commented Nov 12, 2020 at 23:12

### Eigenvectors and eigenvalues

For an equilibrium distribution, you have

$$P \cdot v = \lambda v \quad \text{where \lambda = 1}$$

So you will need to look for the eigenvector $$v$$ that relates to the eigenvalue $$1$$.

You can compute it by solving the following homogeneous system of linear equations

$$(P-\lambda) v = \begin{bmatrix} \frac{1}{2}-\lambda&\frac{1}{4}&\frac{1}{8}&\frac{1}{8} \\ \frac{1}{4}&\frac{1}{8}-\lambda&\frac{1}{8}&\frac{1}{2} \\ \frac{1}{8}&\frac{1}{8}&\frac{1}{2}-\lambda&\frac{1}{4} \\ \frac{1}{8}&\frac{1}{2}&\frac{1}{4}&\frac{1}{8}-\lambda \\ \end{bmatrix} v = 0$$

which you could do by writing the matrix in row echelon form with Gaussian elimination

### Sudoku

Maybe you are supposed to solve it like above, but note how the values are distributed and see how the matrix looks like a sudoku or a magic square (all rows and columns sum up to 1).

Can you use this knowledge to easily think of an eigenvector with eigenvalue 1?

### Solving with computer code

You can solve the above problem automatically and fast with a computer.

m <- rbind(c(4,2,1,1),
c(2,1,1,4),
c(1,1,4,2),
c(1,4,2,1))/8
eigen(m)


which gives


$values 3 1.0000000 0.3952847 0.2500000 -0.3952847 $vectors
[,1]       [,2] [,3]       [,4]
[1,] -0.5  0.6979762 -0.5 -0.1132660
[2,] -0.5  0.1132660  0.5  0.6979762
[3,] -0.5 -0.6979762 -0.5  0.1132660
[4,] -0.5 -0.1132660  0.5 -0.6979762


### Limiting distribution

Does this Markov chain has limiting distribution?

The starting vector can be written as a sum of the eigenvectors

$$x_0 = a_1 v_1 + a_2 v_2 + a_3 v_3 + a_4 v_4$$

Then you will have:

$$\begin{array}{} x_1 &=& P \cdot x_0\\ &=& P \cdot (a_1 v_1 + a_2 v_2 + a_3 v_3 + a_4 v_4) \\&=& a_1 \lambda_1 v_1 + \lambda_2 a_2 v_2 + \lambda_3 a_3 v_3 + \lambda_4 a_4 v_4 \end{array}$$

and you can continue this reasoning to

$$x_n = \lambda_1^n a_1 v_1 + \lambda_2^n a_2 v_2 + \lambda_3^n a_3 v_3 + \lambda_4^n a_4 v_4$$

Now consider the limiting behavior for the values of $$\lambda_i^n$$ and argue which value $$x_n$$ will converge to for $$n \to \infty$$

• equilibrium distribution is the probability vector so all the values should be positive and they all should sum up to 1, but in your solutions there are negative values. Commented Nov 12, 2020 at 15:25
• @bluelagoon you can rescale eigenvectors however you like. The R function scales them such that the length (mean square of the components) equals 1. If $P\cdot v = v$ then also $P \cdot (cv) = (cv)$ Commented Nov 12, 2020 at 15:26