Is the mean (Bayesian) posterior estimate of $\theta$ a (Frequentist) unbiased estimator of $\theta$?

I am wondering about the different ways that Bayesian and Frequentist statistic connect with each other.

I recalled that the Maximum Likelihood estimate of a parameter $$\theta$$ is not necessarily an unbiased estimator of that parameter.

That made me wonder: Is the Mean Posterior estimate of $$\theta$$ an unbiased estimator?

That is,

Does $$\phi(x)=E(\theta\mid x)$$, imply $$E(\phi(x)\mid\theta)=\theta$$?

Note that this is indeed a meaningful question, since $$\phi(x)$$, while it is a Bayesian estimator, is simply a function from the data to the real line and so can also be seen as a classical frequentist estimator.

If this question cannot be answered in general, please assume the prior is uniform.

If not, is there some other Bayesian estimator (i.e. a function from the posterior to $$\mathbb R$$) that is always an unbiased estimator in the frequentist sense?

• Maybe you meant "posterior mean" rather than "mean posterior". Dec 18, 2020 at 1:41
• The posterior mean is biased in favor of values having a higher prior probability. Dec 18, 2020 at 1:42

This is a meaningful question which answer is well-known: when using a proper prior $$\pi$$ on $$\theta$$, the posterior mean $$\delta^\pi(x) = \mathbb{E}^\pi[\theta|x]$$ cannot be unbiased. As otherwise the integrated Bayes risk would be zero: \begin{align*} r(\pi; \delta^\pi) &= \overbrace{\mathbb{E}^\pi\{\underbrace{\mathbb{E}^X[(\delta^\pi(X)-\theta)^2|\theta]}_{\text{exp. under likelihood}}\}}^{\text{expectation under prior}}\\ &= \mathbb{E}^\pi\{\mathbb{E}^X[\delta^\pi(X)^2+\theta^2-2\delta^\pi(X)\theta|\theta]\}\\ &= \mathbb{E}^\pi\{\mathbb{E}^X[\delta^\pi(X)^2+\theta^2]|\theta\}- \mathbb{E}^\pi\{\theta \mathbb{E}^X[\delta^\pi(X)|\theta]\}-\overbrace{\mathbb{E}^X\{\mathbb{E}^\pi[\theta|X]\delta^\pi(X)\}}^{\text{exp. under marginal}}\\ &= \mathbb{E}^\pi[\theta^2]+\underbrace{\mathbb{E}^X[\delta^\pi(X)^2]}_{\text{exp. under marginal}} -\mathbb{E}^\pi[\theta^2]-\mathbb{E}^X[\delta^\pi(X)^2]\\ & = 0 \end{align*} [Notations: $$\mathbb{E}^X$$ means that $$X$$ is the random variable to be integrated in this expectation, either under likelihood (conditional on $$\theta$$) or marginal (integrating out $$\theta$$) while $$𝔼^π$$ considers $$θ$$ to be the random variable to be integrated. Note that $$\mathbb{E}^X[\delta^\pi(X)]$$ is an integral wrt to the marginal, while $$\mathbb{E}^X[\delta^\pi(X)|\theta]$$ is an integral wrt to the sampling distribution.]
A side property of interest is that $$\delta^\pi(x) = \mathbb{E}^\pi[\theta|x]$$ is sufficient in a Bayesian sense, as $$\mathbb{E}^\pi\{\theta|\mathbb{E}^\pi[\theta|x]\}=\mathbb{E}^\pi[\theta|x]$$Conditioning upon $$\mathbb{E}^\pi[\theta|x]$$ is the same as conditioning on $$x$$ for estimating $$\theta$$.
• Thank you. Just a quick question, what exactly does $E^X(\cdot)$ mean? does it mean $E(\cdot|X)$? Nov 12, 2017 at 14:24
• So this would depend on the specific distribution $p(x|\theta)$? Is there no unbiased estimator that is unbiased, regardless of the distribution? Nov 12, 2017 at 14:55
• That's right, but a bayesian estimator will take the correct distribution of $X$ into account, since it will be reflected in the posterior, correct? So thats why my thought was that maybe there is a bayesian estimator that works regardless of the distribution of $X$ Nov 12, 2017 at 16:21
• @SheridanGrant: the "trüe mean" of the observable $\bar{x}$, $\mu_0$, is an unknown and hence varies within a range of possible. Unbiasedness addresses a frequentist property that must hold for all values of $\mu_0$, not just one. In that sense it is impossible to centre the prior at $\mu_0$. Jun 17, 2020 at 17:10