I have read through existing answers on CrossValidated (plus elsewhere online) and can't find what I'm looking for, but do please point me to existing sources if I've missed them.
Let's say I have a data set of N=1000 records, each of which can be manually sampled and labelled as either 'Valid' or 'Invalid' (or True/False, Right/Wrong, etc).
I want to achieve a given level of confidence that all records in the data set are Valid. As I sample records, if I find a single Invalid one I would go back and amend how the data set is created to rectify that and similar problems.
So, after some iterations of spotting Invalids, fixing and recreating the data set, I do some sampling that only includes Valid records. If I want to be (say) 99% or 95% sure that all records are Valid, how big does my sample have to be? (Ideally as a function of N.)
I've tried playing around with Hypergeometric tests (http://en.wikipedia.org/wiki/Hypergeometric_distribution#Hypergeometric_test) - in that context I want to know what k should be, but I don't have a fixed value of K. Rather I want to choose k such that K is likely to be equal to N - but setting K=N obviously works out to a Probability of 1! I'm also wondering if I need to use a Bayesian approach but I don't understand Bayesian stats enough.