Determining the limiting distribution

Question

I just started on a course with stochastic processes, so I am really new to this and I have a problem with the following exercise

---

A Markov chain $X_0, X_1, X_2$ has the transition probability matrix

$$P=\begin{Vmatrix} 0.7 & 0.2 & 0.1 \\ 0 & 0.6 & 0.4 \\ 0.5 & 0 & 0.5\end{Vmatrix}$$

Determine the limiting distribution

---

In my book it says that the limiting distribution $\pi$ is the unique nonnegative solution of the following equations. $$\pi_j=\sum_{k=0}^N \pi_kP_{kj}, \qquad for \qquad j=0,1,...,N \qquad (*)$$ $$\sum_{k=0}^N \pi_k=1 \qquad (**)$$

By this I get that I have to write a system of linear equations and this is the system I get $${\frac {7}{10}\pi_0}+{\frac {2}{5}\pi_1}+{\frac {1}{10}\pi_2}=\pi_0 \qquad (1)$$ $${\frac {0}{10}\pi_0}+{\frac {3}{5}\pi_1}+{\frac {4}{10}\pi_2}=\pi_1 \qquad (2)$$ $${\frac {3}{10}\pi_0}+{\frac {0}{5}\pi_1}+{\frac {1}{2}\pi_2}=\pi_2 \qquad (3)$$ $$\pi_0+\pi_1+\pi_2=1 \qquad (4)$$ I understand that my columns have to sum to 1 because of (*),(**) so in (3) I have 3/10 for $\pi_0$. I solve for (1),(2) and (4) and the solution I get is $\pi_0={\frac{5}{11}}, \pi_1={\frac{3}{11}}, \pi_2={\frac{3}{11}}$ which is not the answer to the exercise in my book. The answer my book gives is that the solution is $\pi_0={\frac{10}{21}}, \pi_1={\frac{5}{21}}, \pi_2={\frac{6}{21}}$ and I can't quite work out what I'm doing wrong?

Answer 1

What needs to sum to 1 is the rows, not the columns, as the process needs to move somewhere from any state (or remain in its state).

Also, the $(1,2)$-element is 0.2, thus $2/10$, not $2/5$.

The stationary distribution can most easily (imo) be computed as the normalized eigenvector corresponding to the unit eigenvalue:

The invariant distribution can be written as $$ \pi ^{\prime }\left( P-I\right) =0^{\prime } $$ or, taking transposes, $$ \left( P^{\prime }-I\right)\pi =0 $$ Hence, $\pi$ is the eigenvector of $P'$ corresponding to the eigenvalue $\lambda _{i}=1$ (the implicit 1 in front of the identity matrix).

In R,

P <- matrix(c(7,0,5,2,6,0,1,4,5),3,3)/10
q <- eigen(t(P))$vectors[,1] 
(q/sum(q)) # normalize sum of probabilities to 1
[1] 0.4761905+0i 0.2380952+0i 0.2857143+0i