If I have a dataset of size $N$, and choose $n$ samples, where $k$ of the samples are OK, how confident can I be that the dataset is OK?

To make it more concrete, suppose I have a dataset D1 with 5,000 values. I choose 20 of these and check each one, finding that all 20 are OK. I have dataset D2 with 5,000 values, I choose 20 of these and find out that 18 of them are OK.

I want to answer the question of "how good is the data", based on the sample that I've taken. Perhaps a better question would be - what sample size do I need to take in order to have a confidence interval of $\pm$ values about the proportion given by the sample? Although I'm not sure what an interval would mean in the case of the sample being all correct. Say I take a sample of 20 from 5000 and they're all correct, I can't then infer that the data has between (4900, 5100) correct samples with some degree of confidence (because the maximum is 5000 of course).

I feel that it's probably something binomial, but am unsure.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.