# Is the mean (Bayesian) posterior estimate of $\theta$ a (Frequentist) unbiased estimator of $\theta$?

I am wondering about the different ways that Bayesian and Frequentist statistic connect with each other.

I recalled that the Maximum Likelihood estimate of a parameter $\theta$ is not necessarily an unbiased estimator of that parameter.

That made me wonder: Is the Mean Posterior estimate of $\theta$ an unbiased estimator?

That is,

Does $\phi(x)=E(\theta|x)$, imply $E(\phi(x)|\theta)=\theta$?

Note that this is indeed a meaningful question, since $\phi(x)$, while it is a Bayesian estimator, is simply a function from the data to the real line and so can also be seen as a classical frequentist estimator.

If this question cannot be answered in general, please assume the prior is uniform.

If not, is there some other Bayesian estimator (i.e. a function from the posterior to $\mathbb R$) that is always an unbiased estimator in the frequentist sense?

This is a meaningful question which answer is well-known: when using a proper prior $\pi$ on $\theta$, the posterior mean $\delta^\pi(x) = \mathbb{E}^\pi[\theta|x]$ cannot be unbiased. As otherwise the integrated Bayes risk would be zero: \begin{align*} r(\pi; \delta^\pi) &= \mathbb{E}^\pi\{\mathbb{E}^X[(\delta^\pi(X)-\theta)^2]|\theta\}\\ &= \mathbb{E}^\pi\{\mathbb{E}^X[\delta^\pi(X)^2+\theta^2-2\delta^\pi(X)\theta]|\theta\}\\ &= \mathbb{E}^\pi\{\mathbb{E}^X[\delta^\pi(X)^2+\theta^2]|\theta\}- \mathbb{E}^\pi\{\theta \mathbb{E}^X[\delta^\pi(X)|\theta]\}-\mathbb{E}^X\{\mathbb{E}^\pi[\theta|X]\delta^\pi(X)\}\\ &= \mathbb{E}^\pi[\theta^2]+\mathbb{E}^X[\delta^\pi(X)^2] -\mathbb{E}^\pi[\theta^2]-\mathbb{E}^X[\delta^\pi(X)^2]\\ & = 0 \end{align*} [Notations: $\mathbb{E}^X$ means that $X$ is the random variable to be integrated in this expectation, while $𝔼^π$ considers $θ$ to be the random variable to be integrated. Note that $\mathbb{E}^X[\delta^\pi(X)]$ is an integral wrt to the marginal, while $\mathbb{E}^X[\delta^\pi(X)|\theta]$ is an integral wrt to the sampling distribution.]
A side property of interest is that $\delta^\pi(x) = \mathbb{E}^\pi[\theta|x]$ is sufficient in a Bayesian sense, as $$\mathbb{E}^\pi\{\theta|\mathbb{E}^\pi[\theta|x]\}=\mathbb{E}^\pi[\theta|x]$$Conditioning upon $\mathbb{E}^\pi[\theta|x]$ is the same as conditioning on $x$ for estimating $\theta$.
• Thank you. Just a quick question, what exactly does $E^X(\cdot)$ mean? does it mean $E(\cdot|X)$? – user56834 Nov 12 '17 at 14:24
• So this would depend on the specific distribution $p(x|\theta)$? Is there no unbiased estimator that is unbiased, regardless of the distribution? – user56834 Nov 12 '17 at 14:55
• That's right, but a bayesian estimator will take the correct distribution of $X$ into account, since it will be reflected in the posterior, correct? So thats why my thought was that maybe there is a bayesian estimator that works regardless of the distribution of $X$ – user56834 Nov 12 '17 at 16:21