I want to find the all the equilibrium distribution of this markov chain Transition matrix

By letting w1= α, where 0<= α <=1, I got w2= (7/15)α w3 = (2/3)α w4= (3/5)α

But I am not sure if its correct.

Also, does this Markov chain has limiting distribution?

  • 3
    $\begingroup$ can you show your calculations as well $\endgroup$ – gunes Nov 12 '20 at 12:47
  • 2
    $\begingroup$ If this is homework, you should probably add the self-study tag as well. $\endgroup$ – chl Nov 12 '20 at 17:54
  • $\begingroup$ Hints: Obviously ergodic because $\mathbf{P}$-matrix has all positive elements. Stationary distribution (4-vector with elements summing to 1) with $\sigma$ with $\sigma\mathbf{P} = \sigma.$ But notice that $\mathbf{P}$ is doubly stochastic (that is columns as well as rows sum to $1).$ $\endgroup$ – BruceET Nov 12 '20 at 23:12

Eigenvectors and eigenvalues

For an equilibrium distribution, you have

$$P \cdot v = \lambda v \quad \text{where $\lambda = 1$}$$

So you will need to look for the eigenvector $v$ that relates to the eigenvalue $1$.

You can compute it by solving the following homogeneous system of linear equations

$$(P-\lambda) v = \begin{bmatrix} \frac{1}{2}-\lambda&\frac{1}{4}&\frac{1}{8}&\frac{1}{8} \\ \frac{1}{4}&\frac{1}{8}-\lambda&\frac{1}{8}&\frac{1}{2} \\ \frac{1}{8}&\frac{1}{8}&\frac{1}{2}-\lambda&\frac{1}{4} \\ \frac{1}{8}&\frac{1}{2}&\frac{1}{4}&\frac{1}{8}-\lambda \\ \end{bmatrix} v = 0$$

which you could do by writing the matrix in row echelon form with Gaussian elimination


Maybe you are supposed to solve it like above, but note how the values are distributed and see how the matrix looks like a sudoku or a magic square (all rows and columns sum up to 1).

Can you use this knowledge to easily think of an eigenvector with eigenvalue 1?

Solving with computer code

You can solve the above problem automatically and fast with a computer.

m <- rbind(c(4,2,1,1),

which gives

 3  1.0000000  0.3952847  0.2500000 -0.3952847

      [,1]       [,2] [,3]       [,4]
 [1,] -0.5  0.6979762 -0.5 -0.1132660
 [2,] -0.5  0.1132660  0.5  0.6979762
 [3,] -0.5 -0.6979762 -0.5  0.1132660
 [4,] -0.5 -0.1132660  0.5 -0.6979762

Limiting distribution

Does this Markov chain has limiting distribution?

The starting vector can be written as a sum of the eigenvectors

$$x_0 = a_1 v_1 + a_2 v_2 + a_3 v_3 + a_4 v_4 $$

Then you will have:

$$\begin{array}{} x_1 &=& P \cdot x_0\\ &=& P \cdot (a_1 v_1 + a_2 v_2 + a_3 v_3 + a_4 v_4) \\&=& a_1 \lambda_1 v_1 + \lambda_2 a_2 v_2 + \lambda_3 a_3 v_3 + \lambda_4 a_4 v_4 \end{array}$$

and you can continue this reasoning to

$$x_n = \lambda_1^n a_1 v_1 + \lambda_2^n a_2 v_2 + \lambda_3^n a_3 v_3 + \lambda_4^n a_4 v_4$$

Now consider the limiting behavior for the values of $\lambda_i^n$ and argue which value $x_n$ will converge to for $n \to \infty$

  • $\begingroup$ equilibrium distribution is the probability vector so all the values should be positive and they all should sum up to 1, but in your solutions there are negative values. $\endgroup$ – bluelagoon Nov 12 '20 at 15:25
  • 1
    $\begingroup$ @bluelagoon you can rescale eigenvectors however you like. The R function scales them such that the length (mean square of the components) equals 1. If $P\cdot v = v$ then also $P \cdot (cv) = (cv)$ $\endgroup$ – Sextus Empiricus Nov 12 '20 at 15:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.