7
$\begingroup$

I know there are methods to calculate a confidence interval for a proportion to keep the limits within (0, 1), however a quick Google search lead me only to the standard calculation: $\hat{p} \pm 1.96*\sqrt\frac{\hat{p}(1-\hat{p})}{N}$. I also believe there is a way to calculate the exact confidence interval using the binomial distribution (example R code would be nice). I know I can use the prop.test function to get the interval but I'm interested in working through the calculation.

Sample situations (N = number of trials, x = number of success):

N=40, x=40
N=40, x=39
N=20, x=0
N=20, x=1
$\endgroup$
  • 1
    $\begingroup$ Check out this paper. It doesn't give the R code you want, but it has many examples of nearly exact confidence intervals. $\endgroup$ – Cyan May 12 '12 at 3:57
6
$\begingroup$

Use a Clopper-Pearson interval?

Wikipedia discribes how to do this here: http://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval

For example if you take your 39 successes in 40 trial example you get:

> qbeta(.025,39,2) #qbeta(alpha/2,x,n-x+1) x=num of successes and n=num of trials
[1] 0.8684141
> qbeta(1-.025,39,2)
[1] 0.9938864

For your 40 out of 40 you get:

> qbeta(1-.025,40,1)    
[1] 0.9993673
> qbeta(.025,40,1)
[1] 0.9119027
$\endgroup$
  • $\begingroup$ The Clopper-Pearson method is excessively conservative when the observed proportion approaches 0 or one. That conservatism is a consequence of them being designed for at least 1-alpha covefrage in the long run rather than good performance for the particular data set being examined. See my answer to this question for more detail; stats.stackexchange.com/questions/8844/… $\endgroup$ – Michael Lew May 13 '12 at 2:09
  • 1
    $\begingroup$ shouldn't it be n-x+1 instead of what you have at the moment/ $\endgroup$ – Algorithmatic Jan 24 at 19:17
10
$\begingroup$

There are many confidence intervals for single proportions and most of them have poor performance for $p$ close to 0 or 1. The "exact" Clopper-Pearson interval mentioned above is very conservative in that setting, meaning that the actual coverage of the interval can be quite a bit larger than the nominal $1-\alpha$.

An interval that has pretty good performance for $p$ close to 0 or 1 is actually the Bayesian credible interval using the Jeffreys prior. See e.g. this paper by Brown, Cai and DasGupta (2002). It is simple to compute in R:

qbeta(c(alpha/2,1-alpha/2),x+0.5,n-x+0.5)

Nevermind that it is Bayesian by nature - it has been shown over and over again to have good frequentist performance!

(Although the Bayesian Jeffreys interval usually is recommended in this setting, it is possible to construct intervals that simultaneously give higher confidence and lower expected length for small $p$; see a recent manuscript of mine.)

$\endgroup$
  • 1
    $\begingroup$ I a point about exact methods that should be noted is that in the hypothesis teting framework they produce a sawtoothed power function. This is because of the discrete nature of random variable like the binomial. It also leads to erratic behavior for confidence intervals. A number of methods that smooth thing out have been proposed. Lots of articles on this in the American Statistician including mine with Christine Liu and a comment on it by Jeff Longmate. $\endgroup$ – Michael R. Chernick May 12 '12 at 14:13
  • $\begingroup$ If the true p is not exactly 0 or 1 you will see this by taking a large sample (unless of course it si something like 0.999999. In small samples getting all successes or all failures may not be a clear indication on the value of p, a general problem with small sample sizes. But if p were equal to 0.5 n doesn't have to be very large for this to be a very unlikely event. If the true p is close to 0 or 1 it would also be worth noting that the variance of the maximum likelihood estimate for p has a much smaller variance than if p were 0.5. $\endgroup$ – Michael R. Chernick May 12 '12 at 14:21
4
$\begingroup$

Why not just do this in a Bayesian way?

That is, set up a beta-distributed prior, and choose some interval whose integral is as big as you want it (working out from the mode, for example).

$\endgroup$
  • $\begingroup$ +1 That is certainly a possibility. But for now I'm interested in the frequentist methods. $\endgroup$ – Glen May 12 '12 at 1:23
  • 1
    $\begingroup$ Concur, the Bayesian approach is very simple for proportions, and gets directly at the salient aspect here which is the confidence interval can't go outside [0,1] and is likely asymmetrical. $\endgroup$ – prototype May 12 '12 at 2:36
4
$\begingroup$

Clopper-Pearson is an exact binomial method and can be used to get confidence intervals for p even when the number of successes is 0 out of N or N out of N. in the first case it will give an interval from 0 to A where A depends on N and alpha and from B to 1 in the latter case.

$\endgroup$
  • 2
    $\begingroup$ Speaking of CIs when the number of successes is 0 (out of N trials), I've always enjoyed the simplicity of the "rule of three" jstor.org/discover/10.2307/… $\endgroup$ – boscovich May 12 '12 at 7:43
  • $\begingroup$ Besides the reference that andrea gives the rule of three is described in van Belle's book "Statistical Rules of Thumb". The book is delightful and very well-written and it explains the many statistical rules of thumb allowing you to understand how they came about and why they make sense. $\endgroup$ – Michael R. Chernick May 12 '12 at 16:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.