Using Rule of Three to obtain confidence interval for a binomial population

Question

I have a large population of data instances (say, 1000 instances) that are either of class1 or of class2. I would like to obtain a confidence interval for how many instances are of class1 without exhaustively checking all instances. I have randomly sampled 50 instances, and all 50 were of class1. I used the rule of three to determine that a 95% confidence interval for the percentage that an instance is of class1 is [0.94, 1].

From my sampling, I know that at least 50 instances are of class1. For the remaining 1000 – 50 = 950 instances whose classes are unknown, I assume I can apply the [0.94, 1] confidence interval found above. Therefore, can I conclude that, with a 95% confidence, there are at least 50 + (1000 – 50)(0.94) = 943 instances from the population of 1000 that are of class1?

If this conclusion isn’t statistically sound, how can I obtain a confidence interval for class1?

Answer 1

The procedure described in the question is intuitive, clear, and accurate.

Problem Formulation

Formally, this is a hypergeometric sampling problem: in a population of $N=1000$ subjects, of which $K$ are in Class 1 and $N-K$ are in Class 2, a sample of size $n=50$ is taken without replacement and it is observed that all $n$ of them are in Class 1. A $95\%$ lower confidence limit $K_{0.95}$ for $K$ is the smallest value that is consistent with these data in the sense that if $K$ were any less than $K_{0.95}$, then the chance that every member of the sample is in Class 1 (as it turned out to be) would be less than $1 - 0.95 = 0.05 = \alpha$, which would be implausible.

Solution

This chance, as a function of the unknown $K$, is easy to compute. Because the sample of $n$ can be taken one at a time, and each time the values of both $K$ and $N$ decrease by $1$, it is equal to the product of the individual chances of observing a subject in Class 1:

$$P(K,n,N) = \frac{K}{N} \times \frac{K-1}{N-1} \times \cdots \times \frac{K-n+1}{N-n+1}.$$

This is a product of a sequence of decreasing fractions. Since $n\ll N$, the obvious bounds (based on replacing each term by the first fraction $K/N$ on the one hand and the first fraction that has been omitted, $(K-n)/(N-n)$, on the other hand) give an excellent approximation:

$$\left(\frac{K-n}{N-n}\right)^n \lt P(K,n,N) \lt \left(\frac{K}{N}\right)^n.$$

The value of $K_{0.95}$ will therefore lie between the solutions $K$ to

$$n\log\left(\frac{K-n}{N-n}\right) \lt \log(\alpha) \lt n\log\left(\frac{K}{N}\right),$$

given by

$$n + (N-n)(1 - 3/n) \approx n + (N-n)(1 + \log(\alpha)/n) \gt K;\\K \gt N \exp(\log(\alpha)/n) \approx N \exp(-3/n).$$

(The appearance of $3$ as the approximation to $-\log(0.05)= 2.9957\ldots$ is the basis for this "Rule of Three".) With $N=1000$ and $n=50$ we have

$$941.764 \lt K_{0.95} \lt 943.082$$

(and these bounds are not appreciably changed by using $3$ instead of $-\log(0.05)$).

The right hand value (upper bound) is the value proposed in the question. In fact, the precise solution is $K_{0.95} = 943$ because

$$P(943, 50, 1000) = 0.04924 \lt 0.05 \le 0.051099 = P(944, 50, 1000).$$