stationary distribution of a markov chain

Question

I am trying to find all the stationary distributions of the $3\times 3$ transition matrix below. because of the third state, the chain isn't irreducible however that isn't a sufficient condition for there not to be a stationary distribution.

I have used $\pi\mathbb{P}=\pi$ to work out $\pi=(\pi_1,\pi_2,\pi_3)$ but I fail to get a solution even with the knowledge that $\pi_1+\pi_2+\pi_3=1.$ I am not sure where I am going wrong.

$$\mathbb{P}=\pmatrix{ 0.4 & 0.6 &0\\ 0.2 & 0.8 &0\\ 0 & 0 & 1\\ }$$

solving the three simultaneous equations, I get $\pi_1=1/3\pi_2$ and $\pi_3=1-2/3\pi_2$ i don't know where to go from there

the fact that the third state is an absorbing state not connected to the other two states is why i have come to a dead end but i am not sure how to find the stationary distribution

Answer 1

Outline:

Your transition matrix shows two non-intercommunicating subclasses $C_1 = \{1,2\}$ and $C_2 = \{3\}.$ So consider two separate chains.

The steady state vector for $C_1$ is $\sigma_1 = c(.75,.25).$ [Show how to derive this; then verify $\sigma_1 C_1 = \sigma_1,$ as required.]

For more complicated, multi-state, irreducible ergodic chains, you may be interested in a computational method of finding steady state vectors. In R, you can find the steady state vector of an irreducible ergodic chain as shown below. (R finds right eigenvectors, hence the transpose t.) The first eigenvector listed in output is proportional to steady state vector. (Code as.numeric strips away complex number notation, irrelevant here, in case some eigenvectors are complex.)

P = matrix(c(.4,.6,.2,.8), nrow=2, byrow=T)
P
     [,1] [,2]
[1,]  0.4  0.6
[2,]  0.2  0.8
eigen(t(P))
eigen() decomposition
$values
[1] 1.0 0.2
$vectors
           [,1]       [,2]
[1,] -0.3162278 -0.7071068
[2,] -0.9486833  0.7071068

g =as.numeric(eigen(t(P))$vec[,1]);  g
[1] -0.3162278 -0.9486833
sg = g/sum(g);  sg
[1] 0.25 0.75
sg%*%P
     [,1] [,2]
[1,] 0.25 0.75

Consequently, if $0\le q\le1,$ then any 3-vector of form $\sigma = ((1-q)\sigma_1,q)$ is steady state for the entire chain. [Verify that $\sigma P = \sigma$ for such a vector.]