0
$\begingroup$

I am trying to find all the stationary distributions of the $3\times 3$ transition matrix below. because of the third state, the chain isn't irreducible however that isn't a sufficient condition for there not to be a stationary distribution.

I have used $\pi\mathbb{P}=\pi$ to work out $\pi=(\pi_1,\pi_2,\pi_3)$ but I fail to get a solution even with the knowledge that $\pi_1+\pi_2+\pi_3=1.$ I am not sure where I am going wrong.

$$\mathbb{P}=\pmatrix{ 0.4 & 0.6 &0\\ 0.2 & 0.8 &0\\ 0 & 0 & 1\\ }$$

solving the three simultaneous equations, I get $\pi_1=1/3\pi_2$ and $\pi_3=1-2/3\pi_2$ i don't know where to go from there

the fact that the third state is an absorbing state not connected to the other two states is why i have come to a dead end but i am not sure how to find the stationary distribution

$\endgroup$
1
$\begingroup$

Outline:

Your transition matrix shows two non-intercommunicating subclasses $C_1 = \{1,2\}$ and $C_2 = \{3\}.$ So consider two separate chains.

The steady state vector for $C_1$ is $\sigma_1 = c(.75,.25).$ [Show how to derive this; then verify $\sigma_1 C_1 = \sigma_1,$ as required.]

For more complicated, multi-state, irreducible ergodic chains, you may be interested in a computational method of finding steady state vectors. In R, you can find the steady state vector of an irreducible ergodic chain as shown below. (R finds right eigenvectors, hence the transpose t.) The first eigenvector listed in output is proportional to steady state vector. (Code as.numeric strips away complex number notation, irrelevant here, in case some eigenvectors are complex.)

P = matrix(c(.4,.6,.2,.8), nrow=2, byrow=T)
P
     [,1] [,2]
[1,]  0.4  0.6
[2,]  0.2  0.8
eigen(t(P))
eigen() decomposition
$values
[1] 1.0 0.2
$vectors
           [,1]       [,2]
[1,] -0.3162278 -0.7071068
[2,] -0.9486833  0.7071068

g =as.numeric(eigen(t(P))$vec[,1]);  g
[1] -0.3162278 -0.9486833
sg = g/sum(g);  sg
[1] 0.25 0.75
sg%*%P
     [,1] [,2]
[1,] 0.25 0.75

Consequently, if $0\le q\le1,$ then any 3-vector of form $\sigma = ((1-q)\sigma_1,q)$ is steady state for the entire chain. [Verify that $\sigma P = \sigma$ for such a vector.]

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.