Is the MLE variance estimator for the normal distribution asymptotically normal? In textbooks, it is mentioned that the maximum likelihood estimators are asymptotically normal. I am having trouble with understanding how this can be true for the estimator of variance (for example, of a normal distribution).
Additionally, Cochran's theorem shows that the unbiased sample variance is chi-square distributed. My intuition is that this applies for finite samples and the chi-square distribution develops to a normal distribution as the sample size approaches infinity.
Can anybody offer an explanation on this matter?
 A: I think I understand now what your concern is.
Let's take $X_i\stackrel{_\text{iid}}{\sim}N(\mu,\sigma^2),\,i=1,2,...,n$
The MLE of $\sigma^2$, $s_n^2$ is $\frac{n-1}{n}$ times the usual unbiased sample estimate, $s_{n-1}^2$. Now we have that $(n-1)s^2_{n-1}/\sigma^2\sim \chi^2_{n-1}$, which is equivalent to $\Gamma(\frac{n-1}{2},2)$ in the shape-scale parameterization.
Consequently, $s^2_{n-1}$ is $\frac{\sigma^2}{n-1}$ times a $\Gamma(\frac{n-1}{2},2)$ and so $s^2_{n}$ is $\frac{\sigma^2}{n}$ times a $\Gamma(\frac{n-1}{2},2)$ and so $s^2_{n}\sim \Gamma(\frac{n-1}{2},\frac{2\sigma^2}{n})$ $-$ again, shape-scale.
This has mean $\frac{n-1}{n}\sigma^2$ and variance $\frac{2(n-1)}{n^2}\sigma^4$, or standard deviation $\frac{\sqrt{2(n-1)}}{n}\sigma^2$. Note, therefore, that the coefficient of variation is $\sqrt{\frac{2}{n-1}}$, so for example, when $n=33$, the mean is $4$ standard deviations above $0$ and when $n=99$ the mean is $7$ standard deviations above $0$.
As long as $n$ is not small, the probability that the normal approximation (the normal distribution with the same mean and variance) associates with negative values will be extremely small $-$ not of practical consequence; by $n=99$ that probability will be about $1.28\times 10^{-12}$, or a bit over $1$ part in a trillion ... and $99$ is quite some way short of the limiting case.

As we can see in the plot, by $n=99$ almost all of the density of the MLE is between $\sigma^2/2$ and $3\sigma^2/2$, and the normal approximation is very close to the true density. The negative half line is many standard deviations from the mean.
