1
$\begingroup$

Say we have a Markov chain with a countably infinite state space, e.g. the non-negative integers.

If we can form and solve equations for the stationary distribution {$\pi_i$}, that satisfies:

$\pi_i = \sum_{j} P_{ji}\pi_j , \ \ \ \ \ i \in \{0,1,2,...\}$

$\sum_{j} \pi_j = 1$

Is it guaranteed that the stationary distribution we've solved for exists?

Or could we solve these equations even if the chain were null recurrent?

$\endgroup$
1
$\begingroup$

If the chain is irreducible then the existence of stationary distribution is guaranteed if and only if all states are positive recurrent.

If the chain is reducible then if there is a positive recurrent component, you can find a stationary distribution for the "sub-chain" of this component, while all other states will get zero. (In this case, there could be multiple solutions).

If non of this conditions is satisfied then there is no stationary distribution, and the set of steady state equations have no solution.

Nir

$\endgroup$
  • $\begingroup$ thanks for the response. I see where you're coming from: viz. an irreducible chain with positive recurrent states is a sufficient condition for a stationary distribution. I am coming from the other direction with i) an irreducible chain, ii) a solution for a stationary distribution; and wondering if these conditions imply the chain is positive recurrent. Or if we could get what looks like a solution to the stationary distribution without the states really being positive recurrent. Thanks! $\endgroup$ – conjectures May 3 '13 at 15:53
  • $\begingroup$ My statement is stronger. If we focus only on irreducible chains, then positive recurrent states is necessary and sufficient condition for a stationary distribution $\endgroup$ – ni6go May 3 '13 at 17:58
  • $\begingroup$ Much appreciated, and sorry to be dense ni6go... would I be right thinking then that if the states were not positive recurrent, I would have been unable to find {$\pi$} that solved both conditions (above)? $\endgroup$ – conjectures May 3 '13 at 19:09
  • $\begingroup$ yes, there will be no solution (again, assuming irreducible chain). $\endgroup$ – ni6go May 4 '13 at 10:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.