1
$\begingroup$

I was trying to get all Stationary Distributions of the following Markov chain. Intuitively, I would say there are two resulting from splitting op the irreducible Markov chain into two reducible ones. However, I feel this is not mathematically correct. How else would I be able to find all stationary distributions? \begin{bmatrix} \frac{1}{3} & \frac{2}{3} & 0 & 0 & 0 \\ \frac{1}{2} & \frac{1}{2} & 0 & 0 & 0 \\ 0 & \frac{1}{2} & 0 & \frac{1}{2} & 0 \\ 0 & 0 & 0 & \frac{1}{4} & \frac{3}{4} \\ 0 & 0 & 0 & \frac{1}{3} & \frac{2}{3} \\ \end{bmatrix}

$\endgroup$
2
  • $\begingroup$ Looking at the limit of $\mathbb P^n$. $\endgroup$
    – Xi'an
    Nov 27 '19 at 14:13
  • $\begingroup$ @Xi'an It is true that this will yield the stationary distributions of the Markov chain, but it is not trivial to compute $P^n$. $\endgroup$
    – Math1000
    Dec 19 '19 at 2:17
0
$\begingroup$

Conditioned on $X_0\in\{0,1\}$ we have from solving \begin{align} \pi_0 &= \frac13\pi_0 + \frac12\pi_1\\ \pi_1 &= \frac23\pi_0 + \frac12\pi_1\\ \pi_0 + \pi_1 &= 1 \end{align} $\pi_0 = \frac37$, $\pi_1=\frac 47$.

Conditioned on $X_0\in\{3,4\}$ we have by solving a similar system of equations $\pi_3 = \frac4{13}$, $\pi_4=\frac9{13}$.

Conditioned on $X_0=2$ we have $\tilde\pi_i = \frac12\pi_i$ for $i\in\{0,1,3,4\}$.

So we have three stationary distributions: those obtained by conditioning on $X_0\in\{0,1\}$ and $X_0\in\{3,4\}$, and the one obtained by conditioning on $X_0=2$: $$ \tilde\pi = \left( \begin{array}{ccccc} \frac{3}{14} & \frac{2}{7} & 0 & \frac{2}{13} & \frac{9}{26} \\ \end{array} \right). $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.