Posterior variance vs variance of the posterior mean

This question is about the frequentist properties of Bayesian methods.

Suppose we have data $${\bf y}$$ generated from a distribution with a single parameter $$\theta$$, equipped with a prior $$\pi(\theta)$$. This leads to a posterior distribution $$\pi(\theta|{\bf y})$$. A Bayesian might characterize the uncertainty surrounding $$\theta$$ using the posterior variance

\begin{align} Var(\theta|{\bf y}) = \int_\Theta \left(\theta-\hat\theta\right)^2\pi(\theta|{\bf y}) d\theta &&&&&& (1) \end{align}

where $$\hat\theta$$ is the posterior mean. A frequentist wearing his Bayesian hat (or perhaps the other way around) might choose to focus on the posterior mean (i.e. an estimate of $$\theta$$) more directly. The variance of the posterior mean is given by

\begin{align} Var(\hat\theta) = \int_\Theta\left(\hat\theta - E(\hat\theta)\right)^2 f(\hat\theta|\theta)d\hat\theta &&&&&& (2) \end{align}

If the posterior mean is unbiased for $$\theta$$, and the sampling distribution $$f(\hat\theta|\theta)$$ "looks like" the posterior distribution $$\pi(\theta|{\bf y})$$, then equations (1) and (2) start to look equivalent.

Why might we care: Thinking about the frequentist coverage of credible intervals (mixing Bayesian and frequentist worlds again), it seems that the ratio of these two quantities should provide valuable information. For example, if the posterior variance is much smaller than the variance of the posterior mean, coverage of the credible intervals may be poor (less than nominal).

1. Does it even make sense to compare these quantities? What kind of information can be gained from comparing them?
2. Can anybody point me to some references dealing with this specific issue? (I've tried searching this, but can't find anything on this site or otherwise dealing with this specific issue).
• A Bayes estimator like a posterior mean is always biased. – Xi'an Sep 20 '19 at 4:46

1 Answer

1. When they're not equivalent, it could be a case of model misspecification.
2. https://arxiv.org/abs/1806.00550
• Hey @ng2410, it is awesome that you added some resources to your answer! But I think it is for the best if you were more specific as to why this resource can help answer the question! – Guilherme Marthe Oct 15 at 19:53