0
$\begingroup$

Consider the transition matrix

$ P = \begin{bmatrix} 1-p&p\\ q&1-q \end{bmatrix} $

for general $2$-state Markov Chain $(0 \le p, q\le 1)$.

  • Find the limiting distribution (if it exists) if $p + q \ne 1$.

Using mathematical induction, it is solved using the following proof:

$ p^n = \frac{1}{p+q} \begin{bmatrix}q&p\\q&p\end{bmatrix} + \frac{(1-p-q)^n}{p+q} \begin{bmatrix}p&-p\\-q&q\end{bmatrix} $

This is totally cumbersome.

Can this be solved in any other way like using $\pi (P-I) = 0$ and so on?

$\endgroup$
0
$\begingroup$

I think induction works well as long as you don't make a dumb mistake on the matrix math (I did the first time I tried it!).

I apologize for the formatting!! Still getting used to latex.

For the first term, we can show that it stays the same after being multiplied by the transition matrix:

$\begin{bmatrix} q&p \\q&p \end{bmatrix} \begin{bmatrix} 1-p & p \\ q & 1-q \end{bmatrix} = \begin{bmatrix} q(1-p) + pq & pq+p(1-q) \\ q(1-p) + pq & pq+p(1-q) \end{bmatrix} $

$\begin{bmatrix} q&p \\q&p \end{bmatrix} \begin{bmatrix} 1-p & p \\ q & 1-q \end{bmatrix} = \begin{bmatrix} q-qp + pq & pq+p-pq) \\ q-qp + pq & pq+p-pq) \end{bmatrix} $

$\begin{bmatrix} q&p \\q&p \end{bmatrix} \begin{bmatrix} 1-p & p \\ q & 1-q \end{bmatrix} = \begin{bmatrix} q & p \\ q & p \end{bmatrix}$

For the second term, it's a little tougher:

$\begin{bmatrix} p & -p \\ -q & q \end{bmatrix} \begin{bmatrix} 1-p & p \\ q & 1-q \end{bmatrix} = \begin{bmatrix} p(1-p) - pq & p^2-p(1-q) \\ -q(1-p) -q^2 & -qp+q(1-q) \end{bmatrix}$

$\begin{bmatrix} p & -p \\ -q & q \end{bmatrix} \begin{bmatrix} 1-p & p \\ q & 1-q \end{bmatrix} = \begin{bmatrix} p(1-p-q) & -p(1-q-p) \\ -q(1-p-q) & q(1-q-p) \end{bmatrix}$

$\begin{bmatrix} p & -p \\ -q & q \end{bmatrix} \begin{bmatrix} 1-p & p \\ q & 1-q \end{bmatrix} = (1-p-q) \begin{bmatrix} p & -p \\ -q & p \end{bmatrix}$

this creates the $(1-p-q)^{n+1}$ term

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.