2
$\begingroup$

I know that in a Markov chain if it is aperiodic and irreducible, the stationary distribution coincides with the limit distribution. But is there anything that guarantees me the existence of the limit distribution? And when the chain is not irreducible, can it still present a stationary distribution? Like for example the chain below. Any reading tips?

enter image description here

Thankful!

$\endgroup$
1
  • $\begingroup$ David, a good reading suggestion is Daniel Gamerman's book, there are some interesting simulations there. $\endgroup$
    – jassis
    Jan 26, 2021 at 18:10

1 Answer 1

4
$\begingroup$

An aperiodic finite-state chain like this one will end up eventually as a Markov Chain on some subset of the states. If the chain is reducible, the subset may depend on the starting value, and on the initial steps taken. The limiting distribution will then be the limiting (and stationary) distribution of the induced chain on the subset of states.

In your case, the chain will either end up on the subset $\{3,5\}$ or $\{1,2\}$. There are two stationary distributions, one with mass on $\{3,5\}$ and one with mass on $\{1,2\}$ (plus mixtures of the two). If it ends up on $\{3,5\}$, the chain will converge to the $\{3,5\}$ limiting distribution; if it ends up on $\{1,2\}$, that one.

Since any finite-state Markov chain will visit any accessible state in finite expected time, we can just agree to forget about what happens before the chain ended up in $\{3,5\}$ or $\{1,2\}$ and treat these as separate problems. If you had infinitely many states, you'd have to worry that it gets more complicated, and I think it does for continuous state spaces; I don't remember whether it does for countably infinite discrete spaces.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.