How to calculate the Expected maximum likelihood variance and mean for gaussian?

Question

I am familiar with the Maximum Likelihood Estimation and Gaussian distribution. I have to big question from Bishop pattern recognition 2006 book.

1. As it is written on page 27, It says that the maximum likelihood underestimates variance.
2. It shows that the correct value for variance which equals to:

$$ E[\sigma_{ml}^{2}] = (\frac{N - 1}{N})\sigma^2 $$

I have know idea how it is calculated and didn't get its point and intuition. Would someone tell me what the above equation really is and talk about its application?

Answer 1

The ML (maximum likelihood) estimate for the variance is $$\sigma^2_{ml}=\frac{1}{n}\sum_{i=1}^n (X_i-\bar X)^2$$ where $\bar X=\frac{1}{n}\sum_{j=1}^n X_j$. Given formula is the expected value of this expression, assuming $E[X_i]=\mu$ and $\operatorname{var}(X_i)=\sigma^2$ and independent samples. It's a bit tedious but you can find the derivation here.

It underestimates the variance because $\frac{N-1}{N}\sigma^2<\sigma^2$.