When looking back at the Metropolis-Hastings algorithms, I think I have been missing the most important point:
How do Metropolis-Hastings algorithms ensure the constructed MC has a limiting distribution same for all initial distributions?
From what I have read, the algorithms try to construct a MC that has the target distribution being a stationary distribution and with respect to which the MC is reversible. However, a stationary distribution may not be the limiting distribution, and it is the limiting distribution only when the limiting distribution exists. For example, for a finite state MC, the limiting distribution exists if and only if the MC is irreducible and aperiodic. However, I haven't seen a description of the Metropolis-Hastings algorithms that states explicitly how existence of the limiting distribution can be ensured by properly choosing the proposal/auxiliary MC and accept/reject probabilities at each iteration.