This is an extension to this previous question, and is related to exercise 4.7 from Gelman et al.'s BDA3.

When is the Bayesian posterior mean $m(y) \equiv E[\theta \mid y]$ unbiased for $\theta$, considering theta either as a random variable (Bayesian perspective) or fixed (Frequentist perspective)?


marked as duplicate by Xi'an bayesian Aug 21 at 23:35

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.


Take the Bayesian perspective, supposing that $E[m(y)\mid\theta] = \theta$ (posterior mean is unbiased). Then

$$E[\theta m(y)] = E[E[\theta m(y) \mid \theta]] = E[\theta E[m(y)\mid\theta]] = E[\theta^2].$$

But also

$$E[\theta m(y)] = E[E[\theta m(y) \mid y]] = E[m(y) E[\theta\mid y]] = E[m(y)^2].$$


$$E[(m(y)-\theta)^2] = E[m(y)^2] + E[\theta^2] - 2E[\theta m(y)] = 0$$

which implies that $m(y) = \theta$ almost surely. This can occur when, e.g., the prior is a point mass at $\mu_0$.

Take the frequentist perspective, where $\theta = \mu_0$ and unbiasedness means

$$E[m(y)\mid\theta] = E[m(y)\mid\mu_0] = E[m(y)] = \mu_0.$$

Then one can construct a prior that yields an unbiased posterior, e.g. if $y_i \sim_\text{iid} N(\theta, \sigma^2)$ and we suppose $\theta \sim N(\mu_0, \sigma_0^2)$ then the posterior mean is the well-known weighted average

$$m(y) = \frac{\sigma_0^2}{\sigma_0^2 + \sigma^2/n}\bar{y} + \frac{\sigma^2/n}{\sigma_0^2 + \sigma^2/n}\mu_0$$

which has expectation $\mu_0$ when the truth is $\theta = \mu_0$. Here $m(y) \ne \theta = \mu_0$ almost surely.

However, in this setting the model is wrong (specifically, the prior). The true prior would be a point mass at $\mu_0$, giving us $m(y) = \mu_0$, which aligns with the Bayesian perspective. Thus, the Bayesian model posterior mean may be nondegenerate and unbiased when the model is wrong!


Not the answer you're looking for? Browse other questions tagged or ask your own question.