aperiodic property and the existence of limiting or stationary distribution for Markov chain

Question

Given a Markov chain, what are the relationship between the property of aperiodic and the existence of stationary distribution or limiting distribution? Moreover, if a Markov chain is claimed to be aperiodic, can I ensure that all the states will have self-loop arc?

Answer 1

There is no implication relation between aperiodicity and stationarity. Take the examples of an aperiodic and transient Markov chain and of a positive recurrent periodic Markov chain. The only sure things are that (i) if a chain is periodic, with period $\rho$, it does not converge to the stationary distribution when the later exists, as the probability of visiting a set $A$ starting from $x$ at time $t$ will depend on $t\text{ mod }\rho$. And (ii) an irreducible aperiodic Markov chain on a finite state space is recurrent (and hence has a stationary distribution).

I do not understand what you mean by self-loop arc. If you mean that there is a probability to remain in the same state, this is not correct. An aperiodic Markov chain may enjoy zeroes on the diagonal of the transition matrix $Q$. For instance, take this matrix $$Q = \begin{pmatrix} 0 & a & 0 & b \\ \frac{1}{2} & 0 & \frac{1}{3}+c & d \\ 0 & a & 0 & b \\ e & 0 & f & 0 \end{pmatrix} $$ Then the associated Markov chain is aperiodic whenever $a,b,c,d,e,f>0$ and the rows sum up to one. (Going back to state 1 is possible in two steps as $(1,2,1)$ and three steps as $(1,2,4,1)$. The largest common denominator of $2$ and $3$ being one, the period is $1$.)