This question is about the frequentist properties of Bayesian methods.

Suppose we have data ${\bf y}$ generated from a distribution with a single parameter $\theta$, equipped with a prior $\pi(\theta)$. This leads to a posterior distribution $\pi(\theta|{\bf y})$. A Bayesian might characterize the uncertainty surrounding $\theta$ using the posterior variance

\begin{align} Var(\theta|{\bf y}) = \int_\Theta \left(\theta-\hat\theta\right)^2\pi(\theta|{\bf y}) d\theta &&&&&& (1) \end{align}

where $\hat\theta$ is the posterior mean. A frequentist wearing his Bayesian hat (or perhaps the other way around) might choose to focus on the posterior mean (i.e. an estimate of $\theta$) more directly. The variance of the posterior mean is given by

\begin{align} Var(\hat\theta) = \int_\Theta\left(\hat\theta - E(\hat\theta)\right)^2 f(\hat\theta|\theta)d\hat\theta &&&&&& (2) \end{align}

If the posterior mean is unbiased for $\theta$, and the sampling distribution $f(\hat\theta|\theta)$ "looks like" the posterior distribution $\pi(\theta|{\bf y})$, then equations (1) and (2) start to look equivalent.

Why might we care: Thinking about the frequentist coverage of credible intervals (mixing Bayesian and frequentist worlds again), it seems that the ratio of these two quantities should provide valuable information. For example, if the posterior variance is much smaller than the variance of the posterior mean, coverage of the credible intervals may be poor (less than nominal).

  1. Does it even make sense to compare these quantities? What kind of information can be gained from comparing them?
  2. Can anybody point me to some references dealing with this specific issue? (I've tried searching this, but can't find anything on this site or otherwise dealing with this specific issue).
  • 1
    $\begingroup$ A Bayes estimator like a posterior mean is always biased. $\endgroup$ – Xi'an Sep 20 '19 at 4:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.