3
$\begingroup$

I would like to know what will happen to the sample variance (not sample mean variance) as sample size increases. I tried to google but all of them are about sample mean variance rather than sample variance.

My thoughts: it will increase because more samples are taken which means there will be more differences?

$\endgroup$
  • $\begingroup$ What if you would think of the variance as the average distance to the mean? $\endgroup$ – Mr Tsjolder Sep 11 '18 at 6:12
3
$\begingroup$

There are two main estimates of the variance. There's the maximum likelihood (ML) estimator $\hat{\sigma}^2_{ML}=\frac{1}{n}\underset{i=1}{\overset{n}{\sum}}(x_i-\bar{x})^2$ and there's the unbiased estimator $\hat{\sigma}^2=\frac{1}{n-1}\underset{i=1}{\overset{n}{\sum}}(x_i-\bar{x})^2$. The ML is biased since it has expectation $E(\hat{\sigma}^2_{ML})=\frac{n-1}{n}\sigma^2$, whereas the other estimator is unbiased.

However, for the question at hand, this distinction between the two estimators is not of interest, since both converge to $\sigma^2$ as the sample size goes to infinity. That is, $\underset{n\rightarrow\infty}{\lim}\hat{\sigma}^2_{ML} = \sigma^2$ and $\underset{n\rightarrow\infty}{\lim}\hat{\sigma}^2 = \sigma^2$.

So as the sample size grows, the closer your estimated variance will be to the true variance.

Another way of thinking of this is that if you have observed all observations in the population, you will know the true variance. If you observe all observations except one, you will still have a extremely good estimate. If you have only a few observations, not so much. So the more observations, the better your estimate will be.

| cite | improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ Wouldn't the unbiased estimator for the sample variance be $\frac{1}{n-1}\sum_{i=1}^n $ instead? $\endgroup$ – Emil Sep 11 '18 at 8:57
  • $\begingroup$ Yes. That is correct. A typo on my behalf. I will correct my answer. Thank you for pointing this out. $\endgroup$ – Phil Sep 11 '18 at 10:48
  • 1
    $\begingroup$ Although an analysis of the expectation of the sample variance may be sort of relevant, it does not answer the question about what happens to the sample variance itself, even when you assume--as you have implicitly done here--that the underlying distribution has a finite variance. $\endgroup$ – whuber Sep 11 '18 at 14:30
  • $\begingroup$ Yes, I am indeed assuming finite variance. Since the sample variance converges to the the population variance (when it exists), then the variance of the variance estimate should go towards zero. Or am I misunderstanding you, @whuber? $\endgroup$ – Phil Sep 11 '18 at 14:45
  • $\begingroup$ See my comments to the answer by user2974951. $\endgroup$ – whuber Sep 11 '18 at 15:57
2
$\begingroup$

A simple simulation shows that for the standard normal distribution the sample variance approaches the population variance and doesn't change significantly with different sample sizes (it varies around 1 but not by much).

enter image description here

Edit: it's true that the t-distribution is a little different, the sample variance of the t-distribution is equal to $\dfrac{v}{v-2}$, where $v$ is the degrees of freedom and $v>2$. So for degrees of freedom 2 or less the variance varies wildly, since it's not strictly defined.

enter image description here

enter image description here

enter image description here

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Could you clarify if your intended standard deviation in the simulation is 1? $\endgroup$ – ReneBt Sep 11 '18 at 8:02
  • 1
    $\begingroup$ The data was generated from the standard normal distribution, so the population variance is equal to 1 (the expected value). $\endgroup$ – user2974951 Sep 11 '18 at 8:05
  • $\begingroup$ What would your simulation show if the data were drawn from a Student t distribution with 3 degrees of freedom? Or, worse yet, with 2 or fewer df? $\endgroup$ – whuber Sep 11 '18 at 14:17
  • 1
    $\begingroup$ @whuber I added your examples, you made a good point, sample variance converges when it is defined. $\endgroup$ – user2974951 Sep 11 '18 at 15:52
  • $\begingroup$ It's subtler than that: the sample variance will converge only provided its variance is finite. That is equivalent to the distribution having a finite fourth moment. The first distribution I suggested has a finite variance but infinite fourth moment. Such distributions are often used for modeling returns in financial markets, for instance, so the issue is not merely theoretical. Even for df between 2 and 4 the sample variance will suddenly jump occasionally and not settle down to any asymptotic value--it's just harder to see in your plots. $\endgroup$ – whuber Sep 11 '18 at 15:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.