Consider the transition matrix:
$\begin{bmatrix} \frac{1}{5} & \frac{4}{5} & 0 & 0 & 0 \\ \frac{1}{2} & \frac{1}{2} & 0 & 0 & 0 \\ \frac{1}{5} & \frac{1}{5} & \frac{1}{5} & \frac{1}{5} & \frac{1}{5} \\ 0 & \frac{1}{3} & \frac{1}{3} & \frac{1}{3} & 0 \\ 0 & 0 & 0 & 0 & 1 \end{bmatrix}$

What is the expected long run proportion of time the chain spends at $a$, given that it starts at $b$.

I know that I must use the stationary distributions of each $\pi(j)$ in question. Since $a$ and $b$ only communicate with each other, I get the system of simulataneous equations:

  • $\pi(a) = \frac{1}{2} \cdot \pi(b) + \frac{1}{5} \cdot \pi(a)$

  • $\pi(b) = \frac{4}{5} \cdot \pi(a) + \frac{1}{2} \cdot \pi(b)$

with these I am getting a distribution $\pi = (\frac{5}{13}, \frac{8}{13})$, Is this correct?

if the distribution started at $c$, as in the title of the post, would my equations now be 3 simulatenous equations which look like:

  • $\pi(a) = \frac{1}{5} \cdot \pi(a) + \frac{1}{2} \cdot \pi(b) + \frac{1}{5} \cdot \pi(c)$

  • $\pi(a) = \frac{4}{5} \cdot \pi(a) + \frac{1}{2} \cdot \pi(b) + \frac{1}{5} \cdot \pi(c)$

  • $\pi(c) = \frac{1}{5} \cdot \pi(c)$

I am uncertain about the last equation. What I am confused about is $c$ leads to every state, but if I include all of them then I will have a system of 6 equations. Since the question is asking specifically about $a$ which can only be reached by states $a,b,c$, shouldn't we only be considering the equations I wrote?


What is the expected long run proportion of time the chain spends at $a$, given that it starts at $b$?

This exercise, technically, asks for the limiting probability value $\ell_b(a)$. You can note that the limiting distribution $\ell_b= \left(\frac{5}{13}, \frac{8}{13}, 0, 0, 0\right)$ that you correctly evaluated is also a stationary distribution of the given matrix -- a limiting distribution will always be stationary. That matrix although is a reducible one and so it can have more than one stationary distribution. In fact a second one is $(0, 0, 0, 0, 1)$.

Now, the question that you made in the title is about the limiting distribution $\ell_c$, and, of course, specifically about its first value $\ell_c(a)$:

$$\ell_c(a) = \lim_{n \to \infty}P(X_n=a | X_0= c) = \frac{5}{7}\frac{5}{13} = \frac{25}{91}$$

If I didn't get it wrong this is a self-study question, so I will leave to you to find the middle steps of this solution. Consider that $\ell_c$ has non-zero values only for $a$, $b$ and $e$, it is, in fact, a weighted sum of the two stationary distributions above (and therefore, it is a stationary distribution as well).

| cite | improve this answer | |
  • $\begingroup$ I placed the bounty -- not the OP. This was posted over 3 years ago, so I don't think you have to worry about the fact that it's a homework question. $\endgroup$ – The Pointer May 24 at 23:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.