# What's the accuracy of data obtained through a random sample?

I'm a newbie at stats, so if I make any mistaken assumptions here please tell me.

There's a population N of people. (For example N can be 1,000,000.) Some of the people are redheads. I take a sample n of people (say 10,) and find that j of them are redheads.

What can I say about the general proportion of redheads in the population? I mean, my best approximation is probably j/n, but what would be the standard deviation of that approximation?

By the way, what is the accepted term for this?

-
Why do we always pick on the gingers? :) –  Brandon Bertelsen Apr 11 '11 at 0:18

You can think of this as a binomial trial -- your trials are sampling "redhead" or "not readhead". In which case, you can build a confidence interval for your sample proportion ($j/n$) as documented on Wikipedia:

A 95% confidence interval basically says that, using the same sampling algorithm, if you repeated this 100 times, the true proportion would lie in the stated interval 95 times.

Update By the way, I think the term you're looking for might be standard error which is the standard deviation of the sampled proportions. In this case, it's $\sqrt{{p (1-p)} \over {n}}$ where $p$ is your estimated proportion. Note that as $n$ increases, the standard error decreases.

-
@ars: Everything is correct and nicely stated. But one thing seems to be missing: the standard deviation of the "best approximation" j/n depends on the true proportion of redheads, not the estimated one. The problem, of course, is that we don't know the true proportion. But the fact remains that the standard error does not actually equal the standard deviation of the approximation except when the estimate happens to be exactly correct. I know you don't need reminding of this subtlety, nor will most readers, but it's rather relevant to the original question. –  whuber Oct 8 '10 at 2:39
@whuber: good point, thanks for noting that. –  ars Oct 8 '10 at 8:47
@whuber: This clarification left me a bit confused. Given a $j$ and an $n$, what would be the standard error, described by $j$ and $n$? (In contrast to being dependent on the true proportion of redheads, which we can't know.) –  Ram Rachum Oct 8 '10 at 16:42
@cool-RR: ars is correct about the standard error. The point is that the standard error itself is an estimate of how accurate the statistic j/n estimates the true proportion. For example, suppose 10% of all people are redheads. Then in many cases it can happen that j=0 when n=10. You would obtain an SE of Sqrt(0(1-0)/10) = 0. This obviously underestimates the actual precision of your statistic p = j/n = 0/10. The true precision is Sqrt(0.10(1-0.90)/n), even though you don't know that! –  whuber Oct 8 '10 at 17:19
@cool-RR: for small samples, use the Agresti-Coull interval specified in the Wikipedia link on confidence intervals. Based on your observations, you will obtain a 95% interval for estimate. Then, what you will know, based on what you observed, is inherent in the definition of a 95% CI. –  ars Oct 8 '10 at 21:26
show 1 more comment

if your sample size $n$ is not such a tiny fraction of the population size $N$ as in your example, and if you sample without replacement [Sw/oR], a better expression for the [estimated] SE is

$$\hat{SE} = \sqrt{\frac{N - n}{N}\frac{\hat p \hat q}{n}},$$

where $\hat p$ is the estimated proportion $j/n$ and $\hat q = 1- \hat p$.

[the term $\frac{N-n}{N}$ is called the FPC [finite population correction].

altho whuber's remark is technically correct, it seems to suggest that nothing can be done to get, say, a confidence interval for the true proportion $p$. if $n$ is large enough to make a normal approximation reasonable [$np > 10$, say], it is unlikely one would get $j=0$. also, if the sample size is large enough for a normal approximation using the true $SE$ to be reasonable, using $\hat{SE}$ instead also gives a reasonable approximation.

[if your $n$ is really small and you use Sw/oR, you may have to use the exact hypergeometric distribution for $j$ instead of a normal approximation. if you do SwR, the size of $N$ is irrelevant and you can use exact binomial methods to get a CI for $p$.]

in any case, since $p(1-p) \le 1/4$, one could always be conservative and use $\frac{1}{2\sqrt{n}}$ in place of $\sqrt{\frac{\hat p \hat q}{n}}$ in the above. if you do that, it takes a sample of $n = 1,111$ to get an estimated ME [margin of error = 2$\hat {SE}$] of $\pm$.03 [regardless of how big $N$ is!].

-