# How to estimate posterior variance from dependent MCMC samples? Does the usual sample covariance still work?

Suppose we want to estimate posterior variance of $\alpha$ given x, i.e. var($\alpha|x$). We have MCMC posterior samples $a_1,\dots, a_B$, which are not independent. Does $\hat{\text{var}}(\alpha|x) = \frac{1}{B} \sum (a_i-\bar{a})^2$ still serve as a good estimator for var($\alpha|x$)? Since the samples are not independent, we can't get consistency of $\hat{\text{var}}(\alpha|x)$ from law of large number.

In an extreme case of correlated posterior samples, if $a_i$s are all the same, then $\hat{\text{var}}(\alpha|x)$ is definitely not a good estimator.

So in practice how should we estimate posterior variance from MCMC samples?

• – Xi'an May 9 '18 at 8:40
• @Xi'an As I recall, Kipnis and Varadhan talk about variance of Monte Carlo averages, not estimating the variance of the target distribution. – Greenparker May 9 '18 at 15:36

If the Markov chain used in MCMC is Harris ergodic (irreducible, aperiodic, and Harris recurrent), the strong law of large numbers holds for functions with finite expectations. That is for Markov chain samples $X_1, X_2, \dots, X_n$, with stationary distribution $\pi$, and a function $f$ such that $\int f \pi(dx) < \infty$ $$\dfrac{1}{n} \sum_{i = 1}^{n} f(X_i) \overset{a.s.}{\to} \int f \pi(dx) \,.$$