I am familiar with the Maximum Likelihood Estimation and Gaussian distribution. I have to big question from Bishop pattern recognition 2006 book.

  1. As it is written on page 27, It says that the maximum likelihood underestimates variance.
  2. It shows that the correct value for variance which equals to:

$$ E[\sigma_{ml}^{2}] = (\frac{N - 1}{N})\sigma^2 $$

I have know idea how it is calculated and didn't get its point and intuition. Would someone tell me what the above equation really is and talk about its application?


The ML estimate for the variance is $$\sigma^2_{ml}=\frac{1}{n}\sum_{i=1}^n (X_i-\bar X)^2$$ where $\bar X=\frac{1}{n}\sum_{j=1}^n X_j$. Given formula is the expected value of this expression, assuming $E[X_i]=\mu$ and $\operatorname{var}(X_i)=\sigma^2$ and independent samples. It's a bit tedious but you can find the derivation here.

It underestimates the variance because $\frac{N-1}{N}\sigma^2<\sigma^2$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.