Bias of variance/precision estimator using Gamma prior

Question

Assume I have $N$ samples $x_1, \cdots, x_N$ from a Gaussian random variable $X\sim N(\mu, \sigma^2)$ where both $\mu$ and $\lambda = 1/\sigma^2$ are unknown.

If I apply MLE, I have $\mu_{MLE} = \frac{1}{N} \sum_i x_i$, and $\lambda_{MLE}^{-1} = \frac{1}{N} \sum_i (x_i - \mu_{MLE})^2$. Note that $\lambda_{MLE}^{-1}$ is biased, and the unbiased estimate will be $\frac{N}{N-1} \lambda_{MLE}^{-1}$.

Now, I have a Gamma distribution $Gamma(a_0, b_0)$ as a prior for $\lambda$.

With $N$ samples, we have that the posterior of $\lambda$ is $Gamma(a_N, b_N)$ where $a_N = a_0 + N/2$ and $b_N = b_0 + N/2 \lambda_{MLE}^{-1}$.

Obviously, if I choose $a_0=0$ and $b_0=0$, then I reach the MLE solution, which is biased.

My question is, is there any prior for $\lambda$ that gives me an unbiased estimate $\frac{N}{N-1} \lambda_{MLE}^{-1}$? It seems that $a_0=-1/2, b_0=0$ does the job, but that does not define a proper Gamma distribution.

Answer 1

This is a general result of (Blackwell and Girshik, 1954): There is no (proper) prior distribution leading to the posterior expectation being an unbiased estimator.

proof (Blackwell and Girshik, 1954): Assume that the posterior expectation (and Bayes estimate under the squared error loss) $$ \delta(x) = \mathbb{E}[\theta|X=x] $$ is unbiased for $\theta$, i.e. satisfies $$ \mathbb{E}_{\theta_0}[\delta(X)]=\theta_0 $$ for every value of $\theta_0$ (where the expectation is under the conditional distribution of $X$ given $\theta=\theta_0$. Then, by Fubini's theorem, using the joint distribution of $(\theta,X)$, $$ \mathbb{E}[\theta \delta(X)] = \mathbb{E}[\mathbb{E}[\theta|X]\delta(X)] = \mathbb{E}[\delta(X)^2] $$ if we first integrate in $\theta$ and $$ \mathbb{E}[\theta \delta(X)] = \mathbb{E}[\theta\mathbb{E}[\delta(X)|\theta]] = \mathbb{E}[\theta^2]. $$ if we first integrate in $X$. Therefore, the Bayes risk is null, $$ \mathbb{E}[\{\theta - \delta(X)\}^2] = 0, $$ which is impossible (except in the uninteresting settings where the true value of the parameter can be known for sure). $\blacksquare$

---

Naturally, if we remove the condition that the prior is proper, there are many instances of the contrary: it is possible to derive unbiased estimators as posterior expectations with improper priors, as shown in the question for $a_0=-1/2$ and $b_0=0$.