2
$\begingroup$

I am trying to intuitively reconcile the following statement, read from "Probability, Markov Chains, and Queues":

A Markov Chain may possess a stationary distribution but not a limiting distribution. 

This is unintuitive to me. I have written down 4 defenitions/facts that I know that I am trying to use:

1) $\pi$ is a limiting distribution of a Markov Chain with transition matrix $P$ if, for some initial distribution $P(0)$, $\pi = P(0)lim_{n \rightarrow \infty}P^{(n)}$. The elements of $\pi$ need not sum to 1.

2) If, for all valid starting distributions $P(0)$, $P(0)lim_{n \rightarrow \infty}P^{(n)} = \pi$, where $\pi$ is a vector of positive reals summing to 1, then $\pi$ is a steady-state distribution.

3) If a Markov Chain has a steady-state distribution, then it is also the unique stationary distribution.

4) A stationary distribution is a vector $\pi$ of positive reals summing to 1 satisfying $\pi = \pi P$.

So the original statement in question is that there is some vector $\pi$ satisfying (4) for some Markov Chains, but not 1. But fact 2 means that steady state distributions are a subset of limiting distributions, and fact 3 means that steady state distributions are stationary distributions, so how can you have a stationary distribution but not a limiting distribution? Where is my logic wrong?

EDIT: after thinking more, if the statement is correct, the chain has some $\pi = \pi P$ (4) but it is NOT true that the same $\pi$ satisfies $\pi = \pi lim_{n \rightarrow \infty}P^{(n)}$, or else $\pi$ would also be a limiting distribution. I guess this means this chain has some kind of fluctuating P matrix when raised to powers. Maybe related to periodicity.

$\endgroup$
  • $\begingroup$ Is the definition in (1) really what you mean? (That the limit exists for some initial distribution?) Take a stationary distribution $\pi$ from (4) and use this as the initial distribution for your chain. Then $\pi = \pi P^n$, i.e. $\pi$ is automagically a "limiting distribution". $\endgroup$ – P.Windridge May 13 '15 at 16:13
  • $\begingroup$ Edited my question by adding a note. $\endgroup$ – Tommy May 14 '15 at 3:18
  • $\begingroup$ Take the equality $\pi = \pi P$. Right multiply by $P$. You get $\pi P = \pi P^2$. But $\pi P = \pi$. Thus $\pi = \pi P^2$. Repeating gives $\pi = \pi P^n$, for any $n \ge 1$. Take the limit $n\to\infty$ in both sides... $\endgroup$ – P.Windridge May 14 '15 at 9:28
  • $\begingroup$ Are you claiming the statement is false? $\endgroup$ – Tommy May 14 '15 at 15:12
  • $\begingroup$ Yes, with your definitions. I agree with @Brian Borchers below. $\endgroup$ – P.Windridge May 14 '15 at 17:37
5
$\begingroup$

The usual definition of limiting distribution is that a Markov chain has a limiting distribution $\pi$ if for every initial distribution $P(0)$,

$ \lim_{n \rightarrow \infty} P(0)P^{(n)}=\pi $

It's important to get that quantifier right.

A useful example to consider is the Markov chain with

$ P=\left[\begin{array}{cc} 0 & 1 \\ 1 & 0 \end{array} \right]. $

Here, $\pi=[1/2 \;\;1/2]$ is a stationary distribution but not a limiting distribution of the Markov chain. In fact this Markov chain does not have a limiting distribution.

$\endgroup$
  • $\begingroup$ Brian, so then the only difference between a "limiting distribution" and a "steady state distribution" is that steady state must be a valid probability distribution whereas limiting need not sum to 1? $\endgroup$ – Tommy May 15 '15 at 17:13
  • $\begingroup$ In my experience, "limiting distribution" and "steady state distribution" are synonyms, and both assume that $\pi$ sums to 1. $\endgroup$ – Brian Borchers May 15 '15 at 19:11
  • $\begingroup$ The book I mention above gives several examples of limiting distributions that do not sum to 1. $\endgroup$ – Tommy May 15 '15 at 20:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.