I am observing a 1/3 of a large (1 000 000) population (e.g. university students). I know that 10 000 students from my observed sample opted for a extra test. I estimate that 30 000 in total opted for it (10 000 / .33). My question is how do I calculate the confidence interval around my estimation.
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
Assuming all students decide to take the extra test independent from each other you can model the distribution as binomial distribution and calculate the confidence interval for success probability. Wikipedia describes a variety of methods to estimate the CI. The Wald method is the most forward one:
$$\hat{p} \pm z \sqrt{\frac{\hat{p}(1-\hat{p})}{n}},$$
where $z = 1.96$. You can find further discussions about CI for binomial distribution with implementations in R here.
In your example $n = \frac{10^6}{3} \approx 333.333$, the number of positive outcomes is $n_{pos} = 10^5$, so your estimate for the success probability is $\hat{p} = \frac{n_{pos}}{n} \approx 0.03$. Using the formula above we calculate the 95% confidence interval: $[0.02942, 0.03058]$.
Based on this CI you could estimate the number of students of your whole population, number of students taking the extra test $[29421, 30579]$. This is straight forward, it may be more appropriate to use a prediction interval in your case. But this seems less simple than the approach outlines above (see e.g. here).
-
1$\begingroup$ This answer is inferior to better procedures that apply a finite population correction -- which is the entire point of the question! A prediction interval is not a correct solution at all, BTW. With a finite population correction, the confidence interval width will shrink by a factor of $\sqrt{1-1/3}=0.82.$ $\endgroup$– whuber ♦Commented Aug 13, 2020 at 14:21
-
$\begingroup$ @whuber What about the method using the Hypergeometric Distribution? See WRIGHT, Tommy. Exact confidence bounds when sampling from small finite universes: an easy reference based on the hypergeometric distribution. Springer Science & Business Media, 1991 $\endgroup$ Commented Aug 31, 2020 at 14:54
$\begingroup$
$\endgroup$
As Tommy Wright wrote in his book1:
What is more beautiful than a simple and important question with a simple and exact answer that is easy to provide?
https://link.springer.com/content/pdf/bbm%3A978-1-4612-3140-0%2F1.pdf has the code for a SAS macro for generating exact one-sided lower an upper confidence bounds.
1WRIGHT, Tommy. Exact confidence bounds when sampling from small finite universes: an easy reference based on the hypergeometric distribution. Springer Science & Business Media, 1991
answered Nov 25, 2020 at 10:03