Assuming a sample $X_1, X_2, ..., X_n$, the sample variance is calculated as

$s^2 = \frac{1}{n-1} \sum (X_i-\bar{X})^2$

The fact that there is $n-1$ in the denominator instead of $n$ is called the Bessel correction and ensures that $s^2$ is an unbiased estimator of the variance $\sigma_X^2 = \langle X^2 \rangle -\langle X\rangle^2$, that is

$\langle s^2 \rangle = \sigma^2_X$

However, during the derivation of the Bessel correction (e.g. here), one makes use of the fact that the sample elements are independent, thus $\sigma^2_\bar{X} = \sigma_X^2/n$, that is, the sample mean variance is $1/n$ times the $X$ variance.

What happens if this is not the case and the sample elements are actually statistically dependent (correlated)? Since we only have a single sample, we do not actually know what the covariances $\langle X_i X_j\rangle$ would be. How would one estimate $\sigma^2_X$ then from the sample? Is $\langle s^2 \rangle$ still an unbiased estimator and if so, how can I see that?

  • 1
    $\begingroup$ It can be helpful to consider extreme cases. Suppose, for instance, that the $X_i$ were perfectly correlated. For instance, $X_1$ might follow a distribution with unit variance but all the other $X_i$ are set equal to $X_1.$ Your $s^2$ therefore is trying to estimate that underlying unit variance (common to all the $X_i$) but, not matter what, it will always equal $0$ (because there's no variation among the $X_i$ in any sample). $\endgroup$
    – whuber
    Apr 21, 2023 at 15:24
  • $\begingroup$ @whuber, thank you. So essentially the conclusion is that for correlated samples there is no unbiased estimator for the variance without knowing the covariances, right? $\endgroup$
    – Botond
    Apr 24, 2023 at 7:58
  • 1
    $\begingroup$ Under such minimal assumptions we couldn't hope to construct an unbiased estimator, but with additional assumptions I wouldn't want to rule out the possibility that one exists. But yes, it would be extremely helpful to know the correlations or at least to limit their possibilities. (That's one thing that, say, AR models accomplish.) $\endgroup$
    – whuber
    Apr 24, 2023 at 12:19
  • $\begingroup$ @whuber, thank you. $\endgroup$
    – Botond
    Apr 25, 2023 at 12:15


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.