# Standard error of the sampling distribution of the mean

I found an equation that says the standard error of the sampling distribution of the mean is:

$$\sigma_{\bar{X}} = \sigma \cdot \sqrt{\frac{1}{n}-\frac{1}{N}}$$

And when the population size is very large, the factor $1/N$ is approximately equal to zero; and the standard deviation formula reduces to:

$$\sigma_{\bar{X}} = \frac{\sigma}{\sqrt{n}}$$

However, I did not get how he found the first equation, may someone explain it?

I found this equation here: http://stattrek.com/sampling/sampling-distribution.aspx

I could explain the second equation in that way: $$\begin{split} Var(\bar{X}) = Var\left(\frac{\sum_{i=1}^{n} X_i}{n}\right) \\ = \left(\frac{1}{n^2}\right)Var\left(\sum_{i=1}^{n} X_i\right) \\ = \left(\frac{1}{n^2}\right)\left(\sum_{i=1}^{n} Var(X_i)\right) \\ = \left(\frac{1}{n^2}\right)\left(\sum_{i=1}^{n}\sigma^2 \right) \\ = \left(\frac{1}{n^2}\right)\left(n\sigma^2\right) \\ = \frac{\sigma^2}{n} \end{split}$$

However I don't know from where appeared the first equation.

-
possible duplicate of Explanation of finite correction factor – COOLSerdash Jul 31 '14 at 22:22
You should clarify in your question what quantity you're discussing the sampling distribution of, under what circumstances. In particular, you can't rely on links to other pages still being there in the long term. While it's fine to link for context, your question should be able to stand on its own. Take care not to simply copy chunks of text though - brief quotes may be okay. – Glen_b Jul 31 '14 at 23:03
Tried to do some cool algebra and mostly failed, but FWIW: $$\frac{1}{n}-\frac{1}{N}=\left(\frac{nN}{1-\frac 1 N}\right)\cdot\left(\frac{N-n}{N-1}\right)$$The stuff in the rightmost set of parentheses is the FPC², so the stuff in the left parentheses is part of the difference. Dunno if that helps though... – Nick Stauner Jul 31 '14 at 23:26
user53064 How confident are you that it's not FPC? – Glen_b Aug 1 '14 at 4:32

The quoted formula is not quite right. Let's derive the correct one.

Since the population mean (or any other constant) may be subtracted from every value in a population $S$ without changing the variance of the population or of any sample thereof, we might as well assume the population mean is zero. Letting the values in the population be $\{x_i\, \vert\, i\in S\}$, this implies

$$0 = \sum_{i\in S} x_i.$$

Squaring both sides maintains the equality, giving

$$0 = \sum_{i,j\in S}x_ix_j = \sum_{i\in S}x_i^2 + \sum_{i \ne j \in S} x_ix_j,$$

whence

$$\sum_{i\ne j \in S} x_ix_j = -\sum_{i\in S} x_i^2.$$

This key result will be employed later.

Let $S$ have $N$ elements. Because its mean is zero, its variance is the average squared value:

$$s^2 = \frac{1}{N}\sum_{i\in S}x_i^2.$$

(Please note that there can be no dispute about the denominator of $N$; in particular, it definitely is not $N-1$: this is a population variance, not an estimator.)

To find the variance of the sample distribution of the mean, consider all possible $n$-element samples. Each corresponds to an $n$-subset $A\subset S$ and has mean

$$\frac{1}{n}\sum_{i\in A} x_i.$$

Since the mean of all the sample means equals the mean of $S$, which is zero, the variance of these $\binom{N}{n}$ sample means is the average of their squares:

$$s_n^2 = \frac{1}{\binom{N}{n}} \sum_{A\subset S}\left(\frac{1}{n}\sum_{i\in A}x_i\right)^2 = \frac{1}{n^2\binom{N}{n}} \sum_{A\subset S}\sum_{i,j\in A}x_ix_j \\= \frac{1}{n^2\binom{N}{n}} \sum_{A\subset S}\left(\sum_{i\in A}x_i^2 + \sum_{i\ne j\in A}x_ix_j\right) .$$

(Once again, $\binom{N}{n}$, not $\binom{N}{n}-1$, is the correct denominator: this is the variance of a collection of $\binom{N}{n}$ numbers, not an estimator of anything.)

Fix, for a moment, any particular index $i$. The value $x_i$ will appear in $\binom{N-1}{n-1}$ samples, because each such sample supplements $x_i$ with $n-1$ more elements of $S$ out of the $N-1$ remaining elements (sampling is without replacement, remember). Its contribution to the right hand side therefore equals $\binom{N-1}{n-1}x_i^2$.

Also fixing an index $j\ne i$, similar reasoning shows the product $x_ix_j$ appears in $\binom{N-2}{n-2}$ samples, thereby contributing $\binom{N-1}{n-1}x_ix_j$ to the right hand side. Therefore, upon summing over all such $i$ and $j$ in $S$,

$$s_n^2 = \frac{1}{n^2\binom{N}{n}} \left(\binom{N-1}{n-1}\sum_{i\in S}x_i^2 + \binom{N-2}{n-2}\sum_{i\ne j\in S}x_ix_j\right).$$

Plug the first result into that last sum:

$$s_n^2 = \frac{1}{n^2\binom{N}{n}} \left(\binom{N-1}{n-1}\sum_{i\in S}x_i^2 + \binom{N-2}{n-2}\left(-\sum_{i\in S}x_i^2\right)\right).$$

It is now straightforward to relate this to the variance of $S$, because $\sum_{i\in S}x_i^2 = Ns^2$:

$$s_n^2 = \frac{1}{n^2\binom{N}{n}} \left(\binom{N-1}{n-1} - \binom{N-2}{n-2}\right)\left(Ns^2\right) = \frac{s^2}{n}\left(1 - \frac{n-1}{N-1}\right).$$

Thus the sampling variance for sampling with replacment, $\frac{s^2}{n}$, is multiplied by $1 - \frac{n-1}{N-1}$ to obtain the sampling variance for sampling without replacement, $s_n^2$. Accordingly, the multiplicative adjustment for the sampling standard deviation is its square root, $\sqrt{1- \frac{n-1}{N-1}}$. This differs from the quoted formula, which uses $\sqrt{1 - \frac{n}{N}}$.

Two simple checks can give us some comfort concerning the correctness of this result. First, the sample variance of means of samples of size $n=1$, $s_1^2$, obviously equals the population variance $s^2$. The correct formula states

$$s_1^2 = \frac{s^2}{1}\left(1 - \frac{1-1}{N-1}\right) = s^2,$$

as it should. Unfortunately, the quoted formula asserts that $s_1^2 = s^2(\frac{1}{1} - \frac{1}{N})$ which obviously cannot be right. Second, the sample variance of the means of samples of size $n=N$ is zero, because there is no variation, and indeed both formulas give $0$ in this case.

-

This result relates specifically for finite population (of size $N$) and for sampling without replacement (for sample size $n$). It becomes clearer if we write

$$\sigma_{\bar{X}} = \sigma \sqrt{\frac{1}{n}-\frac{1}{N}} = \frac {\sigma}{\sqrt n}\left(\sqrt {1-\frac nN}\right) = \frac {\sigma}{\sqrt n}\left(\sqrt {\frac {N-n}N}\right)$$

Comparing the correction factor with the formula provided in the CV post mentioned in the comment as a possible duplicate-maker of this one, Explanation of finite correction factor, there appears as $\left(\sqrt {\frac {N-n}{N-1}}\right)$. Why this difference in the denominator?

You will notice that some authors use $N$ instead of $N-1$ in the denominator of the FPC; in fact, it depends on whether you work with the sample or population statistic: for the variance, it will be $N$ instead of $N-1$ if you are interested in $S^2$ rather than $\sigma^2$.