Sampling distribution of sample variance of non-normal iid r.v.s

Question

In a first course in statistics we are taught that when we do not know the true variance, we can still perform certain basic tests on the sample mean using the sample variance, $S^2$, instead -- provided we can model the sample using normal random variables. This is because the standardization of the mean $\frac{\bar{X}_n - \mathbb{E}(X)}{S/\sqrt{n}}$ follows a distribution (t-distribution) that is easy to understand.

This rests on the fact that we understand the sampling distribution of the sample variance when the i.i.d random variables modelling the sample are Normal:

If $X_i \sim N(\mu, \sigma^2)$ for $i = 1,2, \ldots n$ then $$\frac{S^2}{\sigma^2/(n-1)^2} \sim \chi^2_{n-1}$$

My question:

• What is the most general thing we can say when the i.i.d random variable $X_i$'s are not normal but have finite mean and variances; in other words is there something analogous to the central limit theorem for sample variance?

Answer 1

The asymptotic distribution for the sample variance (in the general non-normal case) can be found in O'Neill (2014) (Result 14, p. 285). As others have pointed out in the comments to your question, the more general result can be obtained via a combination of the CLT and Slutsky's theorem, working on an expansion for the sample variance (the cited paper has the proof so you can see that technique).

The generalised asymptotic result is similar to the (exact) distribution for the normal case, except that the degrees-of-freedom parameter is affected by the kurtosis of the underlying distribution. Higher kurtosis in the underlying distribution leads to greater accuracy, since tail values are less rare; lower kurtosis leads to less accuracy, since tail values are more rare. As can be seen from Result 14 in the above-cited paper, the general case (with finite variance and kurtosis) has the asymptotic approximation:

$$\frac{S^2}{\sigma^2} \sim \frac{\chi^2 (DF_n)}{DF_n} \quad \quad \quad DF_n \equiv \frac{2 \sigma^4}{\mathbb{V}(S^2)} = \frac{2n}{\kappa - (n-3)/(n-1)},$$

where $\kappa$ is the kurtosis of the underlying distribution. In the case of a mesokurtic distribution (such as the normal distribution) you have $\kappa = 3$, which gives $DF_n = n-1$, which is the well-known distribution for the normal case. (You have accidentally squared this term in the equation in your question.) In the case of an underlying platykurtic (leptokurtic) distribution, the degrees-of-freedom is higher (lower) than in the normal case.

As you can see from the definition of the degrees-of-freedom parameter in this result, this parameter is formed from the underlying kurtosis through the variance of the sample variance. (The kurtosis affects the variance of the sample variance, so that is why it enters into this analysis.) The degrees-of-freedom parameter is adjusted to ensure that the variance of the chi-squared distribution matches the true variance of the sampling statistic.