The expected long run proportion of time the chain spends at $a$ , given that it starts at $c$

Question

Consider the transition matrix:
$\begin{bmatrix} \frac{1}{5} & \frac{4}{5} & 0 & 0 & 0 \\ \frac{1}{2} & \frac{1}{2} & 0 & 0 & 0 \\ \frac{1}{5} & \frac{1}{5} & \frac{1}{5} & \frac{1}{5} & \frac{1}{5} \\ 0 & \frac{1}{3} & \frac{1}{3} & \frac{1}{3} & 0 \\ 0 & 0 & 0 & 0 & 1 \end{bmatrix}$

What is the expected long run proportion of time the chain spends at $a$, given that it starts at $b$.

I know that I must use the stationary distributions of each $\pi(j)$ in question. Since $a$ and $b$ only communicate with each other, I get the system of simulataneous equations:

• $\pi(a) = \frac{1}{2} \cdot \pi(b) + \frac{1}{5} \cdot \pi(a)$

• $\pi(b) = \frac{4}{5} \cdot \pi(a) + \frac{1}{2} \cdot \pi(b)$

with these I am getting a distribution $\pi = (\frac{5}{13}, \frac{8}{13})$, Is this correct?

if the distribution started at $c$, as in the title of the post, would my equations now be 3 simulatenous equations which look like:

• $\pi(a) = \frac{1}{5} \cdot \pi(a) + \frac{1}{2} \cdot \pi(b) + \frac{1}{5} \cdot \pi(c)$

• $\pi(a) = \frac{4}{5} \cdot \pi(a) + \frac{1}{2} \cdot \pi(b) + \frac{1}{5} \cdot \pi(c)$

• $\pi(c) = \frac{1}{5} \cdot \pi(c)$

I am uncertain about the last equation. What I am confused about is $c$ leads to every state, but if I include all of them then I will have a system of 6 equations. Since the question is asking specifically about $a$ which can only be reached by states $a,b,c$, shouldn't we only be considering the equations I wrote?

Answer 1

What is the expected long run proportion of time the chain spends at $a$, given that it starts at $b$?

This exercise, technically, asks for the limiting probability value $\ell_b(a)$. You can note that the limiting distribution $\ell_b= \left(\frac{5}{13}, \frac{8}{13}, 0, 0, 0\right)$ that you correctly evaluated is also a stationary distribution of the given matrix -- a limiting distribution will always be stationary. That matrix although is a reducible one and so it can have more than one stationary distribution. In fact a second one is $(0, 0, 0, 0, 1)$.

Now, the question that you made in the title is about the limiting distribution $\ell_c$, and, of course, specifically about its first value $\ell_c(a)$:

$$\ell_c(a) = \lim_{n \to \infty}P(X_n=a | X_0= c) = \frac{5}{7}\frac{5}{13} = \frac{25}{91}$$

If I didn't get it wrong this is a self-study question, so I will leave to you to find the middle steps of this solution. Consider that $\ell_c$ has non-zero values only for $a$, $b$ and $e$, it is, in fact, a weighted sum of the two stationary distributions above (and therefore, it is a stationary distribution as well).