3
$\begingroup$

I am trying to intuitively reconcile the following statement, read from "Probability, Markov Chains, and Queues":

A Markov Chain may possess a stationary distribution but not a limiting distribution. 

This is unintuitive to me. I have written down 4 defenitions/facts that I know that I am trying to use:

1) $\pi$ is a limiting distribution of a Markov Chain with transition matrix $P$ if, for some initial distribution $P(0)$, $\pi = P(0)lim_{n \rightarrow \infty}P^{(n)}$. The elements of $\pi$ need not sum to 1.

2) If, for all valid starting distributions $P(0)$, $P(0)lim_{n \rightarrow \infty}P^{(n)} = \pi$, where $\pi$ is a vector of positive reals summing to 1, then $\pi$ is a steady-state distribution.

3) If a Markov Chain has a steady-state distribution, then it is also the unique stationary distribution.

4) A stationary distribution is a vector $\pi$ of positive reals summing to 1 satisfying $\pi = \pi P$.

So the original statement in question is that there is some vector $\pi$ satisfying (4) for some Markov Chains, but not 1. But fact 2 means that steady state distributions are a subset of limiting distributions, and fact 3 means that steady state distributions are stationary distributions, so how can you have a stationary distribution but not a limiting distribution? Where is my logic wrong?

EDIT: after thinking more, if the statement is correct, the chain has some $\pi = \pi P$ (4) but it is NOT true that the same $\pi$ satisfies $\pi = \pi lim_{n \rightarrow \infty}P^{(n)}$, or else $\pi$ would also be a limiting distribution. I guess this means this chain has some kind of fluctuating P matrix when raised to powers. Maybe related to periodicity.

$\endgroup$
5
  • $\begingroup$ Is the definition in (1) really what you mean? (That the limit exists for some initial distribution?) Take a stationary distribution $\pi$ from (4) and use this as the initial distribution for your chain. Then $\pi = \pi P^n$, i.e. $\pi$ is automagically a "limiting distribution". $\endgroup$ May 13, 2015 at 16:13
  • $\begingroup$ Edited my question by adding a note. $\endgroup$
    – Tommy
    May 14, 2015 at 3:18
  • $\begingroup$ Take the equality $\pi = \pi P$. Right multiply by $P$. You get $\pi P = \pi P^2$. But $\pi P = \pi$. Thus $\pi = \pi P^2$. Repeating gives $\pi = \pi P^n$, for any $n \ge 1$. Take the limit $n\to\infty$ in both sides... $\endgroup$ May 14, 2015 at 9:28
  • $\begingroup$ Are you claiming the statement is false? $\endgroup$
    – Tommy
    May 14, 2015 at 15:12
  • $\begingroup$ Yes, with your definitions. I agree with @Brian Borchers below. $\endgroup$ May 14, 2015 at 17:37

1 Answer 1

8
$\begingroup$

The usual definition of limiting distribution is that a Markov chain has a limiting distribution $\pi$ if for every initial distribution $P(0)$,

$ \lim_{n \rightarrow \infty} P(0)P^{(n)}=\pi $

It's important to get that quantifier right.

A useful example to consider is the Markov chain with

$ P=\left[\begin{array}{cc} 0 & 1 \\ 1 & 0 \end{array} \right]. $

Here, $\pi=[1/2 \;\;1/2]$ is a stationary distribution but not a limiting distribution of the Markov chain. In fact this Markov chain does not have a limiting distribution.

$\endgroup$
3
  • $\begingroup$ Brian, so then the only difference between a "limiting distribution" and a "steady state distribution" is that steady state must be a valid probability distribution whereas limiting need not sum to 1? $\endgroup$
    – Tommy
    May 15, 2015 at 17:13
  • $\begingroup$ In my experience, "limiting distribution" and "steady state distribution" are synonyms, and both assume that $\pi$ sums to 1. $\endgroup$ May 15, 2015 at 19:11
  • $\begingroup$ The book I mention above gives several examples of limiting distributions that do not sum to 1. $\endgroup$
    – Tommy
    May 15, 2015 at 20:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.