Unbiased estimator of variance

Previously, I do believe S^2 is an unbiased estimator of σ^2

is a correct conclusion. However, I found the following statement:

This is an example based on simple random sample without replacement. It says S^2 is a biased estimator of σ^2. So I am wondering "S^2 is an unbiased estimator of σ^2" can only be applied to some specific cases? How to understnad this result based on simple random sample?

• Others should be aware that $n$ is the sample size, $N$ is the population size, and the sample is drawn from the finite population without replacement. Sep 17 '16 at 23:22

2 Answers

When sampling from a finite population without replacement, the observations are negatively correlated with each other, and the sample variance $s^2 = \frac{1}{n-1} \sum_i \left( x_i - \bar{x} \right)^2$ is a slightly biased estimate of the population variance $\sigma^2$.

The derivation in this link from Robert Serfling provides a clear explanation of what's going on. The author first proves that if the observations in a sample have constant covariance (i.e. $\mathrm{Cov}\left(x_i, x_j \right) = \gamma$ for all $i\neq j$) that: $$E[s^2] = \sigma^2 - \gamma$$

For independent draws (hence $\gamma = 0$), you have $E[s^2] = \sigma^2$ and the sample variance is an unbiased estimate of the population variance. But the issue you have with sampling without replacement from a finite population is that your draws are negatively correlated with each other!

In the case of sampling without replacement from a population of size $N$: $$\text{For i\neq j }\quad \mathrm{Cov}\left(x_i, x_j \right) = \frac{-\sigma^2}{N-1}$$ Hence: $$E\left[s^2\right] = \frac{N}{N-1}\sigma^2$$

I don't know where your statements come from, but it the way you present them they are false. Taking directly the variance of the sample (that is, dividing by n) we get a biased estimator, but using sample variance (dividing by n-1) we get an unbiased estimator.

I think your statement comes from different conflicting sources or your source uses different notations in different parts. Maybe "s" means variance (n) in one page and sample variance (n-1) in the other. The fact that one formula uses "n" with the same meaning the other uses "N" makes me suspect that they aren't consistent.

• Sorry I forget to mention, as Gunns said: "that n is the sample size, N is the population size, and the sample is drawn from the finite population without replacement. " Sep 18 '16 at 4:25