Let say I do an experiment were I get a pass or fail result. I want to know how much single trials (sample size) I should do to get a certain confidence level?

So if the answer that I'm looking for is for example 300 units I got to test, I will test these 300 units and be sure at a confidence level of 95% that my process won't have a failure again.

I started with the poisson law, which could give me the probability that less than 1 fail will happen in the future, but how does the sample size effects the results?


In the scenario you describe, rather than Poisson I would read this as $N=300$ independent observations of a binary outcome [pass/fail]. This would mean that your sampling process has a $Y \sim $ Binomial$(N, p)$ distribution, where $p$ is the probability of a failure.


$P(Y \lt 1 | N, p) = (1-p)^N = 0.95$

$N = \ln(0.95)/ \ln (1-p)$

While this is the relationship between sample size and the chance of getting a failure it's not actually very helpful. Because it's easy to imagine that whatever the failure rate, if we take a big enough sample we're almost sure to get a failure. I.e. $P(Y \lt 1 | N, p)$ goes to zero with $N$.

Instead, I suppose you have an idea or an agreement what $p$ should be at most; e.g. say $p=0.01$. If that's the case you could calculate the largest acceptable number of of failures by

qbinom(0.95, 300, 0.01) #6

But this doesn't tell you what sample size to use.

If you want to do a sample size calculation you'd need to additionally specify how accurate you need the estimated proportion $p$ to be. Then you could use the formulae in [1].

[1] http://www.itl.nist.gov/div898/handbook/prc/section2/prc242.htm


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.