I'm working with automating some data-retrieval, and now need to do some verification of the results. The system has automatically flagged 6% of the records, and I would like to determine a reasonable size of the remaining records to be manually verified, which will give me some assurance that other errors, if present, are identified.
Here's the quick numbers:
- 2,502 total records
- 152 (6%) known errors
Let's say I want an 80% chance of spotting errors in the remaining ~2,350 records. How many should I request be reviewed? Is there an easy formula to calculate it with different thresholds?
Update: Thanks for the input, everyone.
My goal is to determine a rough number of "good" records which should be manually reviewed to determine if there are still errors. These are all results which were labeled as "high confidence" by the automated system we used, so in theory we expect them to be 100% accurate.
I'm trying to be diligent in not fully trusting the automated process, but also conscientious of the cost of reviewing these. I would estimate that review will take approximately 1 minute per result; so reviewing all 2500 would be 40 hours of labor.