# 10% rule for sample sizes

In an introductory stats book by Nicole Radziwell "Statistics the easy way with R" , an assumption used for nearly every statistical test (e.g.t-tets, anova, etc) is that the sample size should not be more than 10% of the population size (where the population size in known).

I havent seen it mentioned in other introductory stats books. My question is: how important is theis assumotion and what is the reason for it?

The author didn't say where this comes from (at least as far as I could see). A bit of googling suggests that it refers to making a simple random sample without replacement, if the sample size is above 10% then the observations wont be (approximately) independent, and indepedence of observations is an assumption stated in most textbooks.

In the case where you have a small population (e.g. <50), should you give up on this assumption because it results in sample sizes too small to get decent estimates of variability and to check other assumptions such as normality.

Thank you for any insights.

In statistical models that use parameters for the underlying distributions, these parameters correspond to aspects of the empirical distribution of an infinite population (called a "superpopulation"). Thus, for statistical tests and confidence intervals that deal with model parameters, we are implicitly making inferences about a quantity relating to an infinite population. If we instead wish to make inferences about quantities relating to a finite population this is usually done by imposing an adjustment to the standard tests and confidence intervals called a finite population correction (FPC).

When we have a finite population of $$N \in \mathbb{N}$$ units, the FPC term "disappears" as $$N \rightarrow \infty$$, reflecting the fact that this term is an "adjustment" from the case where $$N = \infty$$. Moreover, in most applications, the FPC term tends to be determined by the proportion of sampled values --- as this approaches zero the term "disappears" from the relevant equations. The author of the book you are reading probably takes the view that when the proportion of sampled values in the population is less than 10%, the FPC adjustment is small enough that it can be safely ignored, whereas when it is larger than 10% it is large enough that it should not be ignored. This is an arbitrary demarcation, and I don't really see any sense in it. In my view, it is much better just to use the FPC when you are making inferences about a finite population.

An applied example: Suppose you observe $$n$$ data points and want to obtain a confidence interval for the mean of a population. If you use the standard confidence interval for the mean parameter of the underlying distribution (implicitly the mean of an infinite superpopulation) then your interval has the form:

$$\text{CI}(1-\alpha) = \Bigg[ \bar{x} \pm \frac{t_{n-1,\alpha/2}}{\sqrt{n}} \cdot s \Bigg].$$

However, we can add a "finite population correction" term to this formula to obtain a confidence interval for the mean of a finite population of $$N$$ units:

$$\text{CI}(1-\alpha) = \Bigg[ \bar{x} \pm \sqrt{\frac{N-n}{N}} \cdot \frac{t_{n-1,\alpha/2}}{\sqrt{n}} \cdot s \Bigg].$$

You can see that the FPC term is a multiplicative term equal to the square-root of the unsampled proportion of values in the population. As $$N \rightarrow \infty$$ the unsampled proportion approaches one and so the FPC term "disappears". You can also see that this latter formula gives you a confidence interval that will allow for any sampling proportion, so it is not necessary to come up with a "rule of thumb" for how low the sampling proportion should be.

Now, when we sample 10% of the population, the FPC term is $$\sqrt{0.9} \approx 0.9487$$, and evidently the author of your book thinks that this is close enough to one that it can be safely ignored (but if it is higher than this it should not be ignored). The author of your book is essentially asserting a "rule-of-thumb" --- if $$n/N \leqslant 0.9$$ then $$FPC \geqslant 0.9487$$, which means you can take $$FPC=1$$ without serious error. As I said above, my preference would be to avoid any such rule and simply use the FPC term when making an inference for a finite population.

• This is a great post. It really explains where the rule of thumb comes from and goes further to say how to deal with the situation when the sample proportion is large. I had one followup question. This site says the condition isn't normally checked for testing differences between two means except for small populaitons or very large samples, but is more imortant for testing differences of proportions. Why would that be so? statisticshowto.datasciencecentral.com/10-condition – Robert S. Apr 11 '19 at 23:21
• I don't think agree that there is any different when testing proportions. In that context, the observed variables are binary indicators, but the FPC term manifests the same way. So long as you are testing differences of proportions where the sample sizes are each a small proportion of the respective populations, the FPC term will be close to one, and therefore close to "disappearing". – Ben Apr 12 '19 at 5:57

If you're sampling a finite population without replacement, you're not sampling independently; your new observations in the sample avoid previously sampled cases.

This is generally a good thing!

However if you're using calculations based on assuming independence you will overestimate variances (instead of the formulas for the sampling you're doing) this will impact properties of CIs and tests. On the other hand if your sample is a very small fraction of the population this barely makes a difference. A common rule of thumb that people use is to ignore it if the standard deviation is overestimated by less than:about 5%.

This corresponds to the 10% rule you mention.

Also see finite population correction.