Variance of sample Variance

Question

In page 292 of Introduction to Mathematical Statistics by Hogg and Craig it is stated that in order for the variance of the sample variance, i.e. $\text{var}(S^2)$ to exist we need to assume that $E[X^4]<\infty$, that is the fourth (uncentered) moment of our RV is finite. My question is, how do these two connect?

I tried to derive a formula for the variance of the sample variance that does not reveal the connection as far as I can see. Since we know that $S^2=\frac{1}{n-1} \sum_{i=1}^n \left( X_i-\bar{X} \right)^2$ is an unbiased estimator of $\sigma^2$, i.e. the population variance, what we need to find is $$E\left[ \left( S^2-\sigma^2 \right) \right]^2$$ which upon simplifying becomes $$E\left(S^4\right)-\sigma^4$$.

From now on the algebra will get horrific though and I am not even sure whether we can proceed without stating the distribution of $X$. Is there perhaps another way to make sense of this statement? Thank you in advance.

Answer 1

Note that $S^2$ has terms involving $X_i^4$, and so $E[S^2]$ is the sum of terms involving $E[X_i^4]$. Thus, if the fourth moment is not finite, neither is $E[S^2]$ finite, nor is var$(S^2)$ finite. Some people say that various quantities such as expectations, variances, etc must be said to be undefined when the corresponding integrals/sums diverge. Others reserve the term "undefined" for cases when the integrals/sums lead to indeterminate forms such as $\infty - \infty$. The latter group would say that for a Cauchy random variable $X$, $E[X]$ is undefined, $E[X^2]$ is defined (but unbounded), and var$(X)$ is undefined (since $E[X]$ is undefined and so var$(X) = E[X^2] - (E[X])^2$ makes no sense). The former group would say that $E[X]$, $E[X^2]$, and var$(X)$ are all undefined for a Cauchy random variable.