5
$\begingroup$

So I know that a connected Markov chain has a stationary distribution $\pi$ that satisfies

$$\lim_{t \rightarrow \infty} a_t = \pi,$$

where $a_t$ is the average probability distribution at time $t$. So this average converges to our stationary distribution. However, does that mean that $\lim_{t \rightarrow \infty} p_t = \pi$ (where $p_t$ is the probability vector of where the Markov chain is after $t$ steps)?

Just going off of the average definition, couldn't it be that the Markov chain bounces around two or more different probability distributions, so that the average is still $\pi$, but $\pi$ itself is never a probability distribution for the states of the Markov chain?

For example, say we have 3 states and $\pi = (0.2, 0.3, 0.5)$, then $p_{(1)} = (0.1, 0.2, 0.7)$ and $p_{(2)} = (0.3, 0.4, 0.3)$. Say for even $i$, $p_i = p_{(1)}$, and for uneven $i$, $p_i = p_{(1)}$, which means that the distribution of the process oscillates between $p_{(1)}$ and $p_{(2)}$. So the average $\lim_{t \rightarrow \infty} a_t$ is indeed $\pi$, but not $\lim_{t \rightarrow \infty} p_t = \pi$, since $p_t$ always switches between $p_{(1)}$ and $p_{(2)}$.

This is obviously a simple example, but I hope you understand my general question.

$\endgroup$
1
  • 1
    $\begingroup$ I think you need to add aperiodic to guarantee stationarity. Now if there exists a stationarity distribution, it is necessarily unique. $\endgroup$
    – Xi'an
    Jul 31, 2023 at 16:05

1 Answer 1

4
$\begingroup$

Here is a fairly general statement from Meyn & Tweedie (1994) that addresses the question: convergence to the invariant probability measure holds for all Harris recurrent aperiodic chains (without aperiodicity it fails!)

enter image description here

with an additional result about the unicity of the stationary measure provided by recurrence only

enter image description here

and another one about the convergence of the averages

enter image description here

where$$S_n(f)=\sum_{i=1}^n f(\it\Phi_i)$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.