21
$\begingroup$

The variance is said to be a measure of spread. So, I had thought that the variance of 3,5 is equal to the variance of 3,3,5,5 since the numbers are equally spread. But this is not the case, the variance of 3,5 is 2 while the variance of 3,3,5,5 is 1 1/3.

This puzzles me, given the explanation that variance is supposed to be a measure of spread.

So, in that context, what does measure of spread mean?

$\endgroup$
  • 3
    $\begingroup$ This is a nice question but the title is not search-friendly, unless somebody happens to be searching for the same data set as you! I wonder if "Why does the variance of a sample change if the observations are duplicated?", or similar, would sum up the more general problem? $\endgroup$ – Silverfish Jun 20 '15 at 23:41
  • $\begingroup$ Silverfish's suggested title change is a good one; I urge you to consider using it. $\endgroup$ – Glen_b Jun 21 '15 at 15:13
  • $\begingroup$ @Silverfish Thanks for the suggestion, I have changed the title accordingly. $\endgroup$ – René Nyffenegger Jun 22 '15 at 14:32
29
$\begingroup$

If you define variance as $s^2_{n}=$$\,\text{MSE}\,$$=\frac1n \sum_{i=1}^n (x_i-\bar{x})^2$ -- similar to population variance but with sample mean for $\mu$, then both your samples would have the same variance.

So the difference is purely because of Bessel's correction in the usual formula for the sample variance ($s^2_{n-1}=\frac{n}{n-1}\cdot \text{MSE}=\frac{n}{n-1}\cdot \frac1n \sum_{i=1}^n (x_i-\bar{x})^2=\frac{1}{n-1}\sum_{i=1}^n (x_i-\bar{x})^2$, which adjusts for the fact that the sample mean is closer to the data than the population mean is, in order to make it unbiased (taking the right value "on average").

The effect gradually goes away with increasing sample size, as $\frac{n-1}{n}$ goes to 1 as $n\to\infty$.

There's no particular reason you have to use the unbiased estimator for variance, by the way -- $s^2_n$ is a perfectly valid estimator, and in some cases may arguably have advantages over the more common form (unbiasedness isn't necessarily that big a deal).

Variance itself isn't directly a measure of spread. If I double all the values in my data set, I contend they're twice as "spread". But variance increases by a factor of 4. So more usually, it is said that standard deviation, rather than variance is a measure of spread.

Of course, the same issue occurs with standard deviation (the usual $s_{n-1}$ version) as with variance -- when you double up the points the standard deviation changes, for the same reason as happens with the variance.

In small samples the Bessel correction makes standard deviation somewhat less intuitive as a measure of spread because of that effect (that duplicating the sample changes the value). But many measures of spread do retain the the same value when duplicating the sample; I'll mention a few --

  • $s_n$ (of course)

  • the mean (absolute) deviation from the mean

  • the median (absolute) deviation from the median

  • the interquartile range

$\endgroup$
  • 2
    $\begingroup$ "There's no particular reason you have to use the unbiased estimator" -- indeed you shouldn't necessarily estimate anything. The variance of {3, 5} itself is 1, per the first formula. As you point out, the questioner has attempted to estimate the variance of a population from which this is presumed to be a sample, but who knows whether it is or not. $\endgroup$ – Steve Jessop Jun 21 '15 at 12:44
1
$\begingroup$

As some sort of mnemonic, $V\,X = E\,V\,X + V\,E\,X$. So the expected value of a sample's variance is too low, with the difference being the variance of the sample's mean.

The usual sample variance formula compensates for that, and the variance of the sample's mean scales inversely with sample size.

As an extreme example, taking a single sample will always show a sample variance of 0, obviously not indicating a variance of 0 for the underlying distribution.

Now for 2 and 4 evenly weighted samples, the corrective factors are $2/1$ and $4/3$, respectively. So your calculated expected variances differ by a factor of $2/3$. The variance of the sample itself is $1$ in either case. But the first case presents a weaker case for $4$ being the mean of the base distribution, and every other value would mean a larger variance.

$\endgroup$
  • 1
    $\begingroup$ By conflating estimators with statistics, this answer confuses, rather than clarifies, the question. Please read Glen_b's original answer in this thread. The argument in the first two paragraphs is mysterious because it seems to be irrelevant to the question. $\endgroup$ – whuber Jun 20 '15 at 14:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.