0
$\begingroup$

Is the following "vague" statement correct, and if so are there a good reference out there which formally work this out?

For a given initial distribution $\vec{x}$, any finite space, discrete time Markov chain described by the transition matrix $P$ will in the limit of many iterations $n$ either

  1. converge to a fixed distribution $\lim_{n \to \infty} P^{n} \vec{x}$ (which may depend on the choice of initial distribution $\vec{x}$ itself).

  2. or exhibit periodic behaviour with some period $d>1$. With periodic behaviour we mean that the state of the Markov chain "jumps" between $d$ different limiting distributions given by $\{ \lim_{n \to \infty} P^{dn} \vec{x}, \lim_{n \to \infty} P^{dn+1} \vec{x}, ..., \lim_{n \to \infty} P^{dn+(d-1)} \vec{x}\}$.

I have read through some material of Markov chains, but I never encountered such an explicit statement. Also, does a similar statement hold for general space, discrete time Markov chains?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ This is more complex because there can exist several ergodic sets, some being cyclic (with possibly different perdiods) some being regular. See Chap 2 of the classical book Finite Markov Chains by Kemeny and Snell. There are some variants in the vocabulary. $\endgroup$
    – Yves
    Commented Jan 11, 2021 at 8:02

1 Answer 1

Reset to default
1
$\begingroup$

This from MIT Open Courseware has the discussion of discrete-space results I think you want.

Nothing so simple is true for general state spaces, or even for a state space that's a segment of the real line. You can get 'null recurrent' chains that return to a state with probability 1, but not in expected finite time, and which don't have a proper limiting distribution.

You can probably get cycles with many different periods simultaneously (you can with deterministic general-state chains)

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks! But how can I then Interpret their statement in the summary on p. 142: "For an arbitrary finite-state Markov chain, if the initial state is transient, then the Markov chain will eventually enter a recurrent state, and the probability that this takes more than n steps approaches zero geometrically in n; Exercise 3.18 shows how to find the probability that each recurrent class is entered ..." To me this sounds as if the only counter example is the case where one cannot enter the recurrent state in finite number of steps. Is this true? What's an example of this particular Case? $\endgroup$
    – Confinement
    Commented Jan 11, 2021 at 7:40
  • $\begingroup$ For a finite state space, a recurrent state is visited in a finite number of steps with probability one. $\endgroup$
    – Xi'an
    Commented Jan 11, 2021 at 10:59
  • $\begingroup$ For general state spaces Thomas is correct: an irreducible Markov chain is either transient, null recurrent, or positive recurrent. Only the last item corresponds to a limiting distribution. A nice entry on the topic is Meyn & Tweedie 1993 (freely available on-line). $\endgroup$
    – Xi'an
    Commented Jan 11, 2021 at 11:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.