1
$\begingroup$

There are several statistical texts that examine the properties of the sample mean and sample variance for IID normal data. It is well-known that these statistics are unbiased estimators for the true mean and variance parameters. Moreover, for the case of IID normal data, it is well-known that the statistics are independent, and the second-order moments of the statistics are:

$$\mathbb{V}(\bar{X}_n) = \frac{\sigma^2}{n} \quad \quad \quad \mathbb{C}(\bar{X}_n, S_n^2) = 0 \quad \quad \quad \mathbb{V}(S_n^2) = \frac{2 \sigma^4}{n-1}.$$

Question: What is the corresponding generalised result for the second-order moments when the data is IID but may come from another distribution?

$\endgroup$
1
$\begingroup$

Detailed analysis of the moments of the sample moments can be found in O'Neill (2014). This includes all the first and second-order moments of the sample mean and sample variance, their unsampled counterparts, and various components related to these statistics. The general form of the second-order moments depends on the first four moments of the distribution. To facilitate our analysis, we suppose that $X_1,...,X_n$ are IID with mean $\mu$, variance $\sigma^2$, skewness $\gamma$ and kurtosis $\kappa$. The variance of the sample variance exists if $\kappa < \infty$ and does not exist otherwise.

Under these preliminary assumptions, the relevant second-order moments are:

$$\mathbb{V}(\bar{X}_n) = \frac{\sigma^2}{n} \quad \quad \quad \mathbb{C}(\bar{X}_n, S_n^2) = \frac{\gamma \sigma^3}{n} \quad \quad \quad \mathbb{V}(S_n^2) = \bigg( \kappa - \frac{n-3}{n-1} \bigg) \frac{\sigma^4}{n}.$$

It is also worth noting that the correlation between the sample mean and sample variance is:

$$\mathbb{Corr}(\bar{X}_n, S_n^2) = \frac{\gamma}{\sqrt{\kappa - (n-3)/(n-1)}}.$$

The normal distribution is an unskewed mesokurtic distribution, so we have $\gamma = 0$ and $\kappa = 3$, which gives the more specific results in the question. As we can see, in the more general case the sample mean and sample variance will be correlated if the underlying distribution is skewed. If the underlying distribution is positively skewed then they are positively correlated, and if the underlying distribution is negatively skewed then they are negatively correlated.

$\endgroup$
2
  • $\begingroup$ The answer (including derivation) to these questions and much more general ones, have been known for almost a century. See, for instance, Fisher, R. A. (1928), Moments and product moments of sampling distributions, Proceedings of the London Mathematical Society, series 2, volume 30, 199-238 (reprinted in Fisher, R. A. (1950), Contributions to Mathematical Statistics, Wiley: New York). $\endgroup$
    – wolfies
    Nov 26 '21 at 5:36
  • $\begingroup$ @wolfies: Thanks for the additional reference --- I'll check it out. $\endgroup$
    – Ben
    Nov 26 '21 at 6:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.