2
$\begingroup$

I want to perform a test to determine (with 95% confidence) whether at least 70% of a population can perform some task.

The test involves sitting a randomly chosen person down and them attempting a task, which they either pass or fail. Equivalently, I flip a weighted coin.

The tests are expensive, so we will only perform a dozen or so of them.

The problem is to calculate pairs of (sample size, number of passes) that would give the required significance.

I want to be able to say something of the form

"Ask 10 people. If 8 or more pass then you can say with 95 confidence that the true ratio in the population is greater than 70%"

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Do you want to take sampling schemes into account, or do you just want to understand the basic idea of a confidence interval for a binomial distribution? If you want to take real people and do this, and you want to be able to generalize to the population of your country or the world, you will have to work with much more sophisticated analyses. $\endgroup$
    – gung - Reinstate Monica
    Commented Jul 5, 2012 at 16:48
  • $\begingroup$ Yes, it was just about the confidence interval for a binomial distribution, I didn't know that was the correct terminology. $\endgroup$
    – Phil
    Commented Jul 5, 2012 at 17:59

2 Answers 2

Reset to default
4
$\begingroup$

If your subjects are representative of the population and independent of each other then this is just the binomial test. You can get the information from you statement by just doing the tests with different inputs, here is an example using R:

> binom.test(8,10, alt='g', p=0.7)

        Exact binomial test

data:  8 and 10 
number of successes = 8, number of trials = 10, p-value = 0.3828
alternative hypothesis: true probability of success is greater than 0.7 
95 percent confidence interval:
 0.4930987 1.0000000 
sample estimates:
probability of success 
                   0.8

So 8 out of 10 only gives a lower limit of 49% in the 95% confidence interval.

However, you need to expand beyond just these simple statements. With 10 subjects you would need 10 out of 10 successes to say (with 95% confidence) that at least 70% can perform the task. But how likely are you to see 10 out of 10? that depends on the true proportion and you should really do a "what if" analysis with different "true" proportions to see how likely you are to see success.

If you are that limited on how many tests you can do then you may want to look to see if there is any other information available as an estimate that could be used as a prior in a Bayesian analysis.

Improve this answer
$\endgroup$
0
$\begingroup$

Yes you can do this by constructing exact binomial confidence limits for a single proportion or equivalence tolerance limits on the proportion if you take n fixed. If you allow n to be the first time you are able to reach the conclusion you are in the realm of sequential analysis.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.