I just started on a course with stochastic processes, so I am really new to this and I have a problem with the following exercise

A Markov chain $X_0, X_1, X_2$ has the transition probability matrix

$$P=\begin{Vmatrix} 0.7 & 0.2 & 0.1 \\ 0 & 0.6 & 0.4 \\ 0.5 & 0 & 0.5\end{Vmatrix}$$

Determine the limiting distribution

In my book it says that the limiting distribution $\pi$ is the unique nonnegative solution of the following equations. $$\pi_j=\sum_{k=0}^N \pi_kP_{kj}, \qquad for \qquad j=0,1,...,N \qquad (*)$$ $$\sum_{k=0}^N \pi_k=1 \qquad (**)$$

By this I get that I have to write a system of linear equations and this is the system I get $${\frac {7}{10}\pi_0}+{\frac {2}{5}\pi_1}+{\frac {1}{10}\pi_2}=\pi_0 \qquad (1)$$ $${\frac {0}{10}\pi_0}+{\frac {3}{5}\pi_1}+{\frac {4}{10}\pi_2}=\pi_1 \qquad (2)$$ $${\frac {3}{10}\pi_0}+{\frac {0}{5}\pi_1}+{\frac {1}{2}\pi_2}=\pi_2 \qquad (3)$$ $$\pi_0+\pi_1+\pi_2=1 \qquad (4)$$ I understand that my columns have to sum to 1 because of (*),(**) so in (3) I have 3/10 for $\pi_0$. I solve for (1),(2) and (4) and the solution I get is $\pi_0={\frac{5}{11}}, \pi_1={\frac{3}{11}}, \pi_2={\frac{3}{11}}$ which is not the answer to the exercise in my book. The answer my book gives is that the solution is $\pi_0={\frac{10}{21}}, \pi_1={\frac{5}{21}}, \pi_2={\frac{6}{21}}$ and I can't quite work out what I'm doing wrong?

  • $\begingroup$ The matrix equation is $\pi P=\pi$, not $P \pi=\pi$. $\endgroup$ – Xi'an Sep 15 '15 at 12:57

What needs to sum to 1 is the rows, not the columns, as the process needs to move somewhere from any state (or remain in its state).

Also, the $(1,2)$-element is 0.2, thus $2/10$, not $2/5$.

The stationary distribution can most easily (imo) be computed as the normalized eigenvector corresponding to the unit eigenvalue:

The invariant distribution can be written as $$ \pi ^{\prime }\left( P-I\right) =0^{\prime } $$ or, taking transposes, $$ \left( P^{\prime }-I\right)\pi =0 $$ Hence, $\pi$ is the eigenvector of $P'$ corresponding to the eigenvalue $\lambda _{i}=1$ (the implicit 1 in front of the identity matrix).

In R,

P <- matrix(c(7,0,5,2,6,0,1,4,5),3,3)/10
q <- eigen(t(P))$vectors[,1] 
(q/sum(q)) # normalize sum of probabilities to 1
[1] 0.4761905+0i 0.2380952+0i 0.2857143+0i
  • $\begingroup$ I was a little fast on my fingers there on element (1,2), but how do you get that it can be written as that? Is it from * and **? $\endgroup$ – ilhano Sep 16 '15 at 7:41
  • $\begingroup$ it follows from writing * in matrix notation, by collecting the equations for all the $\pi_j$ in row vectors $\pi$ and $\pi P$ (or writing $\pi'$ for a row vector in my answer) $\endgroup$ – Christoph Hanck Sep 16 '15 at 8:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.