Difference between maximum likelihood estimate of variance and its expectation?

Question

I understand that the Maximum Likelihood estimate of variance is given by:

And I understand mathematically how to show that the expectation of the maximum likelihood variance is:

However, I cannot seem to figure out intuitively, why they are different? What is the difference between the maximum likelihood of variance and the expectation of the maximum likelihood of variance? What does the expectation of the ML variance even mean?

What is different in the expectation that leads to the term:

Again I am not looking for the mathematical proof, I am just looking to understand what mechanism is leading to the creation of the bias term.

Answer 1

I think you might be missing the fact that the expected value applies only when you do the sampling over and over, not on any one sample. On any given sample, a bad estimator (not unbiased, not MLE, high variance) might have a really good estimate, but we have no way of knowing by how much any given estimator misses, so we prove properties and take our chances with those estimators that tend to work well.

An estimator is just a way of guessing what the population value is. Maximum likelihood estimators sometimes give good estimates, but every estimator is pretty much always wrong by at least a little bit.

Try simulating some data from $N(0,1)$ and then calculate the variance.

import numpy as np
np.random.seed(2020)
x = np.random.normal(0,1,10)
print(np.var(x, ddof=0)) # This is the variance MLE, ddof=1 is unbiased

You won't get exactly $1$, no matter the random seed, so the estimate is always wrong. What the bias means is that, when you do this simulation thousands of times (a loop, for instance), the average calculated variance will not be $1$ but $0.9$.