It is known that if the transition probability matrix of a Markov Chain is regular, the chain has the positive limiting distribution (limiting distribution with all elements positive).

Does the converse hold? i.e., If a Markov Chain has the positive limiting distribution, is its transition probability matrix regular?

If not, what is an additional condition that is required to make this hold?


If the state space is finite, then regularity is indeed equivalent to the existence of a unique limiting distribution. Let $X=\{X_n:n=0,1,\ldots\}$ be a Markov chain with transition matrix $P_{i,j} = \mathbb P(X_{n+1}=j\mid X_n=i)$ and strictly positive limiting distribution $$\pi_j = \lim_{n\to\infty} \mathbb P(X_n=j\mid X_0=i).$$ This implies that for each pair $(i,j)$ of states, $$ \mathbb P\left(\bigcup_{n=1}^\infty \{X_n=j\mid X_0=i\} \right) = 1, $$ so $X$ is irreducible. If a state $i$ were to have period $d(i)>1$, then $\mathbb P(X_k=i\mid X_0=i)=0$ for $1\leqslant k< d(i)$, and hence $\mathbb P(X_{n\cdot d(i)+k}=i\mid X_0=i)=0$ for $n=1,2,\ldots$ and $1\leqslant k<d(i)$. This contradicts the existence of $\lim_{n\to\infty} \mathbb P(X_n=i\mid X_0=i)$, and so state $i$ is aperiodic. Since periodicity is a class property and $X$ is irreducible, all states are aperiodic.

It follows that if $P^n_{i,j}>0$ then $P^{kn}_{i,j}$ for any positive integer $k$. Let $n_{i,j}=\min\{n: P^n_{i,j}>0\}$ for each pair of states $(i,j)$. Set $$ N = \prod_{i,j=1}^n n_{i,j}, $$ then $P^N_{i,j}>0$ for all $(i,j)$, so that $X$ is a regular Markov chain.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.