Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I found an equation that says the standard error of the sampling distribution of the mean is:

$$\sigma_{\bar{X}} = \sigma \cdot \sqrt{\frac{1}{n}-\frac{1}{N}}$$

And when the population size is very large, the factor $1/N$ is approximately equal to zero; and the standard deviation formula reduces to:

$$\sigma_{\bar{X}} = \frac{\sigma}{\sqrt{n}}$$

However, I did not get how he found the first equation, may someone explain it?

I found this equation here: http://stattrek.com/sampling/sampling-distribution.aspx

I could explain the second equation in that way: \begin{equation} \begin{split} Var(\bar{X}) = Var\left(\frac{\sum_{i=1}^{n} X_i}{n}\right) \\ = \left(\frac{1}{n^2}\right)Var\left(\sum_{i=1}^{n} X_i\right) \\ = \left(\frac{1}{n^2}\right)\left(\sum_{i=1}^{n} Var(X_i)\right) \\ = \left(\frac{1}{n^2}\right)\left(\sum_{i=1}^{n}\sigma^2 \right) \\ = \left(\frac{1}{n^2}\right)\left(n\sigma^2\right) \\ = \frac{\sigma^2}{n} \end{split} \end{equation}

However I don't know from where appeared the first equation.

share|improve this question
2  
possible duplicate of Explanation of finite correction factor –  COOLSerdash Jul 31 at 22:22
2  
You should clarify in your question what quantity you're discussing the sampling distribution of, under what circumstances. In particular, you can't rely on links to other pages still being there in the long term. While it's fine to link for context, your question should be able to stand on its own. Take care not to simply copy chunks of text though - brief quotes may be okay. –  Glen_b Jul 31 at 23:03
    
Tried to do some cool algebra and mostly failed, but FWIW: $$\frac{1}{n}-\frac{1}{N}=\left(\frac{nN}{1-\frac 1 N}\right)\cdot\left(\frac{N-n}{N-1}\right)$$The stuff in the rightmost set of parentheses is the FPC², so the stuff in the left parentheses is part of the difference. Dunno if that helps though... –  Nick Stauner Jul 31 at 23:26
    
user53064 How confident are you that it's not FPC? –  Glen_b Aug 1 at 4:32

2 Answers 2

up vote 4 down vote accepted

The quoted formula is not quite right. Let's derive the correct one.

Since the population mean (or any other constant) may be subtracted from every value in a population $S$ without changing the variance of the population or of any sample thereof, we might as well assume the population mean is zero. Letting the values in the population be $\{x_i\, \vert\, i\in S\}$, this implies

$$0 = \sum_{i\in S} x_i.$$

Squaring both sides maintains the equality, giving

$$0 = \sum_{i,j\in S}x_ix_j = \sum_{i\in S}x_i^2 + \sum_{i \ne j \in S} x_ix_j,$$

whence

$$\sum_{i\ne j \in S} x_ix_j = -\sum_{i\in S} x_i^2.$$

This key result will be employed later.

Let $S$ have $N$ elements. Because its mean is zero, its variance is the average squared value:

$$s^2 = \frac{1}{N}\sum_{i\in S}x_i^2.$$

(Please note that there can be no dispute about the denominator of $N$; in particular, it definitely is not $N-1$: this is a population variance, not an estimator.)

To find the variance of the sample distribution of the mean, consider all possible $n$-element samples. Each corresponds to an $n$-subset $A\subset S$ and has mean

$$\frac{1}{n}\sum_{i\in A} x_i.$$

Since the mean of all the sample means equals the mean of $S$, which is zero, the variance of these $\binom{N}{n}$ sample means is the average of their squares:

$$s_n^2 = \frac{1}{\binom{N}{n}} \sum_{A\subset S}\left(\frac{1}{n}\sum_{i\in A}x_i\right)^2 = \frac{1}{n^2\binom{N}{n}} \sum_{A\subset S}\sum_{i,j\in A}x_ix_j \\= \frac{1}{n^2\binom{N}{n}} \sum_{A\subset S}\left(\sum_{i\in A}x_i^2 + \sum_{i\ne j\in A}x_ix_j\right) .$$

(Once again, $\binom{N}{n}$, not $\binom{N}{n}-1$, is the correct denominator: this is the variance of a collection of $\binom{N}{n}$ numbers, not an estimator of anything.)

Fix, for a moment, any particular index $i$. The value $x_i$ will appear in $\binom{N-1}{n-1}$ samples, because each such sample supplements $x_i$ with $n-1$ more elements of $S$ out of the $N-1$ remaining elements (sampling is without replacement, remember). Its contribution to the right hand side therefore equals $\binom{N-1}{n-1}x_i^2$.

Also fixing an index $j\ne i$, similar reasoning shows the product $x_ix_j$ appears in $\binom{N-2}{n-2}$ samples, thereby contributing $\binom{N-1}{n-1}x_ix_j$ to the right hand side. Therefore, upon summing over all such $i$ and $j$ in $S$,

$$s_n^2 = \frac{1}{n^2\binom{N}{n}} \left(\binom{N-1}{n-1}\sum_{i\in S}x_i^2 + \binom{N-2}{n-2}\sum_{i\ne j\in S}x_ix_j\right).$$

Plug the first result into that last sum:

$$s_n^2 = \frac{1}{n^2\binom{N}{n}} \left(\binom{N-1}{n-1}\sum_{i\in S}x_i^2 + \binom{N-2}{n-2}\left(-\sum_{i\in S}x_i^2\right)\right).$$

It is now straightforward to relate this to the variance of $S$, because $\sum_{i\in S}x_i^2 = Ns^2$:

$$s_n^2 = \frac{1}{n^2\binom{N}{n}} \left(\binom{N-1}{n-1} - \binom{N-2}{n-2}\right)\left(Ns^2\right) = \frac{s^2}{n}\left(1 - \frac{n-1}{N-1}\right).$$

Thus the sampling variance for sampling with replacment, $\frac{s^2}{n}$, is multiplied by $1 - \frac{n-1}{N-1}$ to obtain the sampling variance for sampling without replacement, $s_n^2$. Accordingly, the multiplicative adjustment for the sampling standard deviation is its square root, $\sqrt{1- \frac{n-1}{N-1}}$. This differs from the quoted formula, which uses $\sqrt{1 - \frac{n}{N}}$.


Two simple checks can give us some comfort concerning the correctness of this result. First, the sample variance of means of samples of size $n=1$, $s_1^2$, obviously equals the population variance $s^2$. The correct formula states

$$s_1^2 = \frac{s^2}{1}\left(1 - \frac{1-1}{N-1}\right) = s^2,$$

as it should. Unfortunately, the quoted formula asserts that $s_1^2 = s^2(\frac{1}{1} - \frac{1}{N})$ which obviously cannot be right. Second, the sample variance of the means of samples of size $n=N$ is zero, because there is no variation, and indeed both formulas give $0$ in this case.

share|improve this answer

This result relates specifically for finite population (of size $N$) and for sampling without replacement (for sample size $n$). It becomes clearer if we write

$$\sigma_{\bar{X}} = \sigma \sqrt{\frac{1}{n}-\frac{1}{N}} = \frac {\sigma}{\sqrt n}\left(\sqrt {1-\frac nN}\right) = \frac {\sigma}{\sqrt n}\left(\sqrt {\frac {N-n}N}\right)$$

Comparing the correction factor with the formula provided in the CV post mentioned in the comment as a possible duplicate-maker of this one, Explanation of finite correction factor, there appears as $\left(\sqrt {\frac {N-n}{N-1}}\right)$. Why this difference in the denominator?

@chl answer there mentions

You will notice that some authors use $N$ instead of $N-1$ in the denominator of the FPC; in fact, it depends on whether you work with the sample or population statistic: for the variance, it will be $N$ instead of $N-1$ if you are interested in $S^2$ rather than $\sigma^2$.

...which needs to be reconciled with @whuber's answer.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.