In page 292 of Introduction to Mathematical Statistics by Hogg and Craig it is stated that in order for the variance of the sample variance, i.e. $\text{var}(S^2)$ to exist we need to assume that $E[X^4]<\infty$, that is the fourth (uncentered) moment of our RV is finite. My question is, how do these two connect?

I tried to derive a formula for the variance of the sample variance that does not reveal the connection as far as I can see. Since we know that $S^2=\frac{1}{n-1} \sum_{i=1}^n \left( X_i-\bar{X} \right)^2$ is an unbiased estimator of $\sigma^2$, i.e. the population variance, what we need to find is $$E\left[ \left( S^2-\sigma^2 \right) \right]^2$$ which upon simplifying becomes $$E\left(S^4\right)-\sigma^4$$.

From now on the algebra will get horrific though and I am not even sure whether we can proceed without stating the distribution of $X$. Is there perhaps another way to make sense of this statement? Thank you in advance.

  • $\begingroup$ See: math.stackexchange.com/questions/72975/… $\endgroup$ – wolfies Dec 16 '13 at 14:54
  • $\begingroup$ @wolfies Yeah I've seen that. One of the answers is irrelevant while the other one is too complicated for an intermediate book like the one I am using-and consequently for me. Do you think there is a simpler derivation somewhere? $\endgroup$ – JohnK Dec 16 '13 at 14:59
  • 3
    $\begingroup$ Well, the point is that, since the solution (see link above) is expressed in terms of the first 4 moments of the population, the solution also assumes the existence of those moments. $\endgroup$ – wolfies Dec 16 '13 at 15:58

Note that $S^2$ has terms involving $X_i^4$, and so $E[S^2]$ is the sum of terms involving $E[X_i^4]$. Thus, if the fourth moment is not finite, neither is $E[S^2]$ finite, nor is var$(S^2)$ finite. Some people say that various quantities such as expectations, variances, etc must be said to be undefined when the corresponding integrals/sums diverge. Others reserve the term "undefined" for cases when the integrals/sums lead to indeterminate forms such as $\infty - \infty$. The latter group would say that for a Cauchy random variable $X$, $E[X]$ is undefined, $E[X^2]$ is defined (but unbounded), and var$(X)$ is undefined (since $E[X]$ is undefined and so var$(X) = E[X^2] - (E[X])^2$ makes no sense). The former group would say that $E[X]$, $E[X^2]$, and var$(X)$ are all undefined for a Cauchy random variable.

  • $\begingroup$ Thanks. I think I side with the latter group as there is a little theorem that states that if an expectation for $X_k$ exists then the expectation for all values of $n\leq k$ exist as well. $\endgroup$ – JohnK Dec 16 '13 at 15:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.