# Prove MLE underestimate variance get two different results

$$\hat{\sigma}^2 = \frac{1}{N}\sum_{n = 1}^N (x_n - \bar{x})^2$$ When proving the variance is underestimated using MLE, the following two procedures are generating different results. $$E[\hat{\sigma}^2] = E[\frac{1}{N}\sum_{n = 1}^N (x_n - \bar{x})^2] \\= \frac{1}{N}\sum_{n=1}^N E[(x_n^2 - 2x_n\bar{x} + \bar{x}^2)] \\= \frac{1}{N}\sum_{n=1}^N E[x_n^2 - 2x_n\bar{x} + \bar{x}^2] \\= \frac{1}{N}\sum_{n=1}^N (E[x_n^2]-2\bar{x}E[x_n]+E[\bar{x}^2])\\= \frac{1}{N}\sum_{n=1}^N (E[x_n^2]-2\bar{x}^2+E[\bar{x}^2])\\=E[x_n^2]-2\bar{x}^2+E[\bar{x}^2]$$

$$E[\hat{\sigma}^2] = E[\frac{1}{N}\sum_{n = 1}^N (x_n - \bar{x})^2] \\= \frac{1}{N}E[\sum_{n = 1}^N (x_n^2 - 2x_n\bar{x} + \bar{x}^2)] \\= \frac{1}{N}E[\sum_{n = 1}^N x_n^2 - \sum_{n = 1}^N 2x_n\bar{x} + \sum_{n = 1}^N \bar{x}^2] \\= \frac{1}{N}E[\sum_{n = 1}^N x_n^2 - 2N\bar{x}^2 + N\bar{x}^2]\\=\frac{1}{N}E[\sum_{n = 1}^N x_n^2 - N\bar{x}^2] \\= \frac{1}{N}E[\sum_{n = 1}^N x_n^2] - E[\bar{x}^2] \\= \frac{1}{N}\sum_{n = 1}^N E[x_n^2] - E[\bar{x}^2] \\= E[x_n^2] - E[\bar{x}^2]$$

Can anyone point out which step is wrong in the first procedure?

• Just add some comments here, you should use capital letters here and $E(X_n)=\mu$ not $\bar{X}$ – Deep North Sep 28 '17 at 8:55
• Also, worth pointing out that $\mathbb{E}[\hat \sigma^2] = \sigma^2(1-1/N)$ underestimates the true variance. Hence, you divide by $N-1$ instead of $N$ to get the unbiased estimator. – P.Windridge Sep 28 '17 at 9:40
• I suggest you carefully try to compute first $\mathbb{E}[X_i \bar X]$ and then $\mathbb{E}[\bar X^2]$, remembering that if $i=j$ then $\mathbb{E}[X_i X_j] = \mathbb{E}[X_i^2] \neq \mu^2$! – P.Windridge Sep 28 '17 at 9:47
• @P.Windridge Thanks for pointing out. I made a typo on that in the original post. Corrected now – rifle123 Sep 28 '17 at 15:43

$$E[x^2_n] \neq E[\bar x^2]$$
Second, as pointed out in the comments, you should probably check your notion of expected value. $\bar x$ is an estimator, and $E[x] \neq \bar x$.
To avoid problems in the future, consider that $E[X]$ is the mean and $\bar x$ is the empirical mean. While $\bar x$ is a function of your data, $E[X]$ depends on the distribution of $X$, which you don't really know if you just have a bunch of observations.