4
$\begingroup$

In a first course in statistics we are taught that when we do not know the true variance, we can still perform certain basic tests on the sample mean using the sample variance, $S^2$, instead -- provided we can model the sample using normal random variables. This is because the standardization of the mean $\frac{\bar{X}_n - \mathbb{E}(X)}{S/\sqrt{n}}$ follows a distribution (t-distribution) that is easy to understand.

This rests on the fact that we understand the sampling distribution of the sample variance when the i.i.d random variables modelling the sample are Normal:

If $X_i \sim N(\mu, \sigma^2)$ for $i = 1,2, \ldots n$ then $$\frac{S^2}{\sigma^2/(n-1)^2} \sim \chi^2_{n-1}$$

My question:

  • What is the most general thing we can say when the i.i.d random variable $X_i$'s are not normal but have finite mean and variances; in other words is there something analogous to the central limit theorem for sample variance?
$\endgroup$
  • 2
    $\begingroup$ Yes, there is. The asymptotic distribution of $S^2$ can be derived from the classical CLT and Slutsky's theorem. The asymptotic variance of $S^2$, as you may expect, depends on the fourth moment of $X$. $\endgroup$ – Zhanxiong Dec 1 '17 at 20:07
  • 3
    $\begingroup$ You don't need asymptotics for the denominator -- you need asymptotics for the ratio (the t-statistic itself). You can get an asymptotic result from CLT + Slutsky. Even when both hold it may take quite a large n to be a useful approximation to the null distribution (and there's no guarantee that the asymptotic relative efficiency will be up to much) $\endgroup$ – Glen_b Dec 2 '17 at 4:19
2
$\begingroup$

The asymptotic distribution for the sample variance (in the general non-normal case) can be found in O'Neill (2014) (Result 14, p. 285). As others have pointed out in the comments to your question, the more general result can be obtained via a combination of the CLT and Slutsky's theorem, working on an expansion for the sample variance (the cited paper has the proof so you can see that technique).

The generalised asymptotic result is similar to the (exact) distribution for the normal case, except that the degrees-of-freedom parameter is affected by the kurtosis of the underlying distribution. Higher kurtosis in the underlying distribution leads to greater accuracy, since tail values are less rare; lower kurtosis leads to less accuracy, since tail values are more rare. As can be seen from Result 14 in the above-cited paper, the general case (with finite variance and kurtosis) has the asymptotic approximation:

$$\frac{S^2}{\sigma^2} \sim \frac{\chi^2 (DF_n)}{DF_n} \quad \quad \quad DF_n \equiv \frac{2 \sigma^4}{\mathbb{V}(S^2)} = \frac{2n}{\kappa - (n-3)/(n-1)},$$

where $\kappa$ is the kurtosis of the underlying distribution. In the case of a mesokurtic distribution (such as the normal distribution) you have $\kappa = 3$, which gives $DF_n = n-1$, which is the well-known distribution for the normal case. (You have accidentally squared this term in the equation in your question.) In the case of an underlying platykurtic (leptokurtic) distribution, the degrees-of-freedom is higher (lower) than in the normal case.

As you can see from the definition of the degrees-of-freedom parameter in this result, this parameter is formed from the underlying kurtosis through the variance of the sample variance. (The kurtosis affects the variance of the sample variance, so that is why it enters into this analysis.) The degrees-of-freedom parameter is adjusted to ensure that the variance of the chi-squared distribution matches the true variance of the sampling statistic.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.