# How to calculate confidence interval when only a part of the samples are valid?

I will simplify our problem in this way. Say there are 100,000 cases in total to examine. Due to the time limitation, we randomly selected 2,000 of them. Then we found 1,000 of them are invalid, so we have only 1,000 valid cases left. Finally we categorize these 1,000 valid cases into 2 categories, A and B; they have 300 and 700 cases respectively.

We want to calculate the confidence interval, in order to check the statistical significance of the results. In other words, if the result shows that there are 30% of the cases in category A, how trustworthy this percentage is when talking about the whole population. We used this website, http://www.surveysystem.com/sscalc.htm, to calculate the confidence interval, so the percentages will be like $30\pm 3.7\%$ and $70\pm 3.7\%$.

So there are two ways of deciding the population and the sample size.

(1) Population is 100,000 (real population) and sample size is 2,000;

(2) Population is 50,000 (estimated) and sample size is 1,000.

I think we should use option (2), because we actually found roughly only 50% of the original cases are valid cases. But this ratio is actually estimated by the 2,000 cases we sampled. How does this estimation affect the confidence of the result?

Would somebody recommend other ways to check the statistical significance of the result in our case?

• You want to read up on non-response bias. That may help you elaborate on this question, because to get a suitable answer you need to disclose the nature of the "invalidity" as well what you're really trying to estimate. – whuber Mar 4 '11 at 23:03
• Thanks, whuber. So by "invalid", I meant the cases that are not interesting to our study. For example, assume we study the returned items in a supermarket. We are interested in only the cases where the items are damaged or incomplete. These cases are labeled with this reason, so it is straightforward to extract them from the database. But some of them are mislabeled, i.e. they are actually not because of this reason. So we have read through all of them and exclude those "invalid" cases. – evergreen Mar 4 '11 at 23:18

An auxiliary research question seems to be to estimate the total number of A's and B's in the urn. For this purpose the urn contains only two kinds of balls, black ones and non-black ones, and we want to estimate the number of non-black balls. This is a standard binomial sampling situation. Without more ado, the estimated number of non-black balls equals 100,000 * (1000/2000) = 50,000 and the estimated proportion is 1/2, with standard error $\sqrt{(1/2)(1 - 1/2)/2000}$ = 1.1%. Therefore the estimate of 50,000 has a 99% two-sided confidence interval from 48,560 to 51,440.