Bayesian derivation of unbiased maximum likelihood estimator

Question

I was recently reading an old NIPS paper by Bishop and Qazaz where they claim that an unbiased estimator for variance, based on $N$ Gaussian $\rm i.i.d.$ samples with unknown mean and unknown variance, can be derived by maximizing over the likelihood (data given only the variance) and that this likelihood can be computed by marginalizing out the mean based on a "flat" prior.

It's equations 4-6 in the paper, but to be self-contained, I'm going to quote the paragraphs of interest.

By adopting a Bayesian viewpoint this bias can be removed. The marginal likelihood of $\sigma^2$ should be computed by integrating over the mean $\mu$. Assuming a 'flat' prior $p(\mu)$ we obtain \begin{align} p\left(D\rvert\sigma^2\right)&=\int p\left(D\rvert\sigma^2, \mu\right)p(\mu)d\mu\tag{4}\\ &\propto \frac{1}{\sigma^{N-1}}\exp\left\{-\frac{1}{2\sigma^2}\sum_{n=1}^N\left(z_n-\hat\mu\right)^2\right\}\tag{5}\\ \end{align} Maximizing $(5)$ with respect to $\sigma^2$ then gives $$\tilde \sigma^2 =\frac{1}{N-1}\sum_{n=1}^N\left(z_n-\hat\mu\right)^2\tag{6}$$ which is unbiased.

I've never heard of this before but it's quite interesting.

• Is there another reference for this?
• What is the "flat" prior they are referring to? Is this an improper prior that is proportional to a constant?

The choice of prior here seems to be what makes Equation 5 follow from Equation 4.

Answer 1

Basically you assume a Gaussian prior with variance going to infinity and mean equal to mu-hat. Complete the square and the cross term drops out by definition of mu-hat, and you are left with a standard Gaussian. Take the limit of the prior variance going to infinity at the end (the limit should be sigma/sqrt(N) not sigma/N as shown in the last line, but N can be dropped from the proportionality anyway).