# Find the invariant measure $\pi=(\pi_{1},\pi_{2},\pi_{3})$ for a Markov Chain with transition matrix given

Let $(X_{n})_{n\in\mathbb{N}_{0}}$ be a Markov Chain with state space $M=\left\{x_{1},x_{2},x_{3}\right\}$ and transtition matrix $$\Pi=\left(\begin{array}{ccc}p_{1} & p_{2} & 1-p_{1}-p_{2}\\ q_{1} & q_{2} & 1-q_{1}-q_{2}\\ r_{1} & r_{2} & 1-r_{1}-r_{2}\end{array}\right)$$ where $p_{i}, q_{i},r_{i}\in[0,1]$.

The question: Find the invariant measure $\pi=(\pi_{1},\pi_{2},\pi_{3})$ for this Markov Chain. If $\pi$ is not unique, then show that $\pi$ can be written as a convex combination of the invariant measures that leave invariant some of the recurrence classes, write explicitly this measures.

My attempt: The invariant measura is the solution of equations $$\Pi^{t}\pi=\pi \qquad \mbox{ and }\qquad \pi_{1}+\pi_{2}+\pi_{3}=1.$$ That is equivalent to $$\left(\begin{array}{ccc}p_{1} -1& q_{1} & r_{1}\\ p_{2} & q_{2}-1 & r_{2}\\ 1-p_{1}-p_{2} & 1-q_{1}-q_{2} & -r_{1}-r_{2}\end{array}\right)\left(\begin{array}{c} \pi_{1}\\ \pi_{2} \\ \pi_{3}\end{array}\right)= \left(\begin{array}{c} 0\\ 0 \\ 0\end{array}\right) \qquad \mbox{ and }\qquad \pi_{1}+\pi_{2}+\pi_{3}=1.$$ That is equivalent to $$\left(\begin{array}{ccc}p_{1} -1& q_{1} & r_{1}\\ p_{2} & q_{2}-1 & r_{2}\\ 1 & 1 & 1\end{array}\right)\left(\begin{array}{c} \pi_{1}\\ \pi_{2} \\ \pi_{3}\end{array}\right)= \left(\begin{array}{c} 0\\ 0\\ 1 \end{array}\right).$$ We called $$K=\left(\begin{array}{ccc}p_{1} -1& q_{1} & r_{1}\\ p_{2} & q_{2}-1 & r_{2}\\ 1 & 1 & 1\end{array}\right).$$ Then we have two cases:

• If $\det K \neq 0$, then I do not have problems
• If $\det K = 0$, then I have problems.I do not know how to calculate the measures that leave the recurrence classes invariant, but in this general case I do not know how to calculate their recurrence classes. My idea is to be able to prove that in this case it suffices to consider $\Pi$ as follows $$\Pi=\left(\begin{array}{ccc}p_{1} &1- p_{1} & 0\\ q_{1} & 1-q_{1} & 0\\ 0 & 0 & 1\end{array}\right),\quad \Pi=\left(\begin{array}{ccc}1 & 0 & 0\\ 0 & q_{1} & 1-q_{1}\\ 0 & r_{1} & 1-r_{1}\end{array}\right), \quad \Pi=\left(\begin{array}{ccc}1 & 0 & 0\\ 0 & 1 & 0\\ 0& 0 & 1 \end{array}\right).$$ If I can prove this last, then the problems will be over, but I'm not sure if what I'm saying is true.