1
$\begingroup$

Say we have a Markov chain with a countably infinite state space, e.g. the non-negative integers.

If we can form and solve equations for the stationary distribution {$\pi_i$}, that satisfies:

$\pi_i = \sum_{j} P_{ji}\pi_j , \ \ \ \ \ i \in \{0,1,2,...\}$

$\sum_{j} \pi_j = 1$

Is it guaranteed that the stationary distribution we've solved for exists?

Or could we solve these equations even if the chain were null recurrent?

$\endgroup$

1 Answer 1

2
$\begingroup$

If the chain is irreducible then the existence of stationary distribution is guaranteed if and only if all states are positive recurrent.

If the chain is reducible then if there is a positive recurrent component, you can find a stationary distribution for the "sub-chain" of this component, while all other states will get zero. (In this case, there could be multiple solutions).

If non of this conditions is satisfied then there is no stationary distribution, and the set of steady state equations have no solution.

Nir

$\endgroup$
4
  • $\begingroup$ thanks for the response. I see where you're coming from: viz. an irreducible chain with positive recurrent states is a sufficient condition for a stationary distribution. I am coming from the other direction with i) an irreducible chain, ii) a solution for a stationary distribution; and wondering if these conditions imply the chain is positive recurrent. Or if we could get what looks like a solution to the stationary distribution without the states really being positive recurrent. Thanks! $\endgroup$ May 3, 2013 at 15:53
  • $\begingroup$ My statement is stronger. If we focus only on irreducible chains, then positive recurrent states is necessary and sufficient condition for a stationary distribution $\endgroup$
    – ni6go
    May 3, 2013 at 17:58
  • $\begingroup$ Much appreciated, and sorry to be dense ni6go... would I be right thinking then that if the states were not positive recurrent, I would have been unable to find {$\pi$} that solved both conditions (above)? $\endgroup$ May 3, 2013 at 19:09
  • $\begingroup$ yes, there will be no solution (again, assuming irreducible chain). $\endgroup$
    – ni6go
    May 4, 2013 at 10:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.