I was recently reading an old NIPS paper by Bishop and Qazaz where they claim that an unbiased estimator for variance, based on $N$ Gaussian $\rm i.i.d.$ samples with unknown mean and unknown variance, can be derived by maximizing over the likelihood (data given only the variance) and that this likelihood can be computed by marginalizing out the mean based on a "flat" prior.
It's equations 4-6 in the paper, but to be self-contained, I'm going to quote the paragraphs of interest.
By adopting a Bayesian viewpoint this bias can be removed. The marginal likelihood of $\sigma^2$ should be computed by integrating over the mean $\mu$. Assuming a 'flat' prior $p(\mu)$ we obtain \begin{align} p\left(D\rvert\sigma^2\right)&=\int p\left(D\rvert\sigma^2, \mu\right)p(\mu)d\mu\tag{4}\\ &\propto \frac{1}{\sigma^{N-1}}\exp\left\{-\frac{1}{2\sigma^2}\sum_{n=1}^N\left(z_n-\hat\mu\right)^2\right\}\tag{5}\\ \end{align} Maximizing $(5)$ with respect to $\sigma^2$ then gives $$\tilde \sigma^2 =\frac{1}{N-1}\sum_{n=1}^N\left(z_n-\hat\mu\right)^2\tag{6}$$ which is unbiased.
I've never heard of this before but it's quite interesting.
- Is there another reference for this?
- What is the "flat" prior they are referring to? Is this an improper prior that is proportional to a constant?
The choice of prior here seems to be what makes Equation 5 follow from Equation 4.