30
$\begingroup$

I have used the following r code to estimate the confidence intervals of a binomial proportion because I understand that that substitutes for a "power calculation" when designing receiver operating characteristic curve designs looking at detection of diseases in a population.

n is 150, and the disease, we believe, is 25% prevalent in the population. I have calculated the values for 75% sensitivity and 90% specificity (because that's what people seem to do).

    binom.test(c(29,9), p=0.75, alternative=c("t"), conf.level=0.95)

    binom.test(c(100, 12), p=0.90, alternative=c("t"), conf.level=0.95)

I have also visited this site:

http://statpages.org/confint.html

Which is a java page which calculates binomial confidence intervals, and it gives the same answer.

Anyway, after that lengthy set-up, I want to ask why the confidence intervals are not symmetric, e.g. sensitivity is 

   95 percent confidence interval:
   0.5975876 0.8855583 

   sample estimate probability: 0.7631579

Sorry if this is a stupid question, but everywhere I look seems to suggest that they will be symmetric, and a colleague of mine seems to think they will be too.

|cite|improve this question
$\endgroup$
20
$\begingroup$

They're believed to be symmetric because quite often a normal approximation is used. This one works well enough in case p lies around 0.5. binom.test on the other hand reports "exact" Clopper-Pearson intervals, which are based on the F distribution (see here for the exact formulas of both approaches). If we would implement the Clopper-Pearson interval in R it would be something like (see note):

Clopper.Pearson <- function(x, n, conf.level){
    alpha <- (1 - conf.level) / 2
    QF.l <- qf(1 - alpha, 2*n - 2*x + 2, 2*x)
    QF.u <- qf(1 - alpha, 2*x + 2, 2*n - 2*x)

    ll <- if (x == 0){
          0
    } else { x / ( x + (n-x+1)*QF.l ) }

    uu <- if (x == 0){
          0
    } else { (x+1)*QF.u / ( n - x + (x+1)*QF.u ) }

    return(c(ll, uu))
}

You see both in the link and in the implementation that the formula for the upper and the lower limit are completely different. The only case of a symmetric confidence interval is when p=0.5. Using the formulas from the link and taking into account that in this case $n = 2\times x$ it's easy to derive yourself how it comes.

I personally understood it better looking at the confidence intervals based on a logistic approach. Binomial data is generally modeled using a logit link function, defined as:

$${\rm logit}(x) = \log\! \bigg( \frac{x}{1-x} \bigg)$$

This link function "maps" the error term in a logistic regression to a normal distribution. As a consequence, confidence intervals in the logistic framework are symmetric around the logit values, much like in the classic linear regression framework. The logit transformation is used exactly to allow for using the whole normality-based theory around the linear regression.

After doing the inverse transformation:

$${\rm logit}^{-1}(x) = \frac{e^x}{1+e^{x}}$$

You get an asymmetric interval again. Now these confidence intervals are actually biased. Their coverage is not what you would expect, especially at the boundaries of the binomial distribution. Yet, as an illustration they show you why it is logic that a binomial distribution has asymmetric confidence intervals.

An example in R:

logit <- function(x){ log(x/(1-x)) }
inv.logit <- function(x){ exp(x)/(1+exp(x)) }
x <- c(0.2, 0.5, 0.8)
lx <- logit(x)
upper <- lx + 2
lower <- lx - 2

logxtab <- cbind(lx, upper, lower)
logxtab # the confidence intervals are symmetric by construction
xtab <- inv.logit(logxtab)
xtab # back transformation gives asymmetric confidence intervals

note : In fact, R uses the beta distribution, but this is completely equivalent and computationally a bit more efficient. The implementation in R is thus different from what I show here, but it gives exactly the same result.

|cite|improve this answer
$\endgroup$
  • 2
    $\begingroup$ Did you really mean to say that the logit "transforms the binomial distribution in a normal distribution"?? $\endgroup$ – whuber♦ Nov 19 '10 at 22:06
  • $\begingroup$ @whuber: nice catch of the formula, and nice catch of the formulation. Pretty much not. It makes sure the errors in a logistic regression follow the normal distribution. Thx for the correction. $\endgroup$ – Joris Meys Nov 20 '10 at 0:18
  • $\begingroup$ Just a brief technical note, the "arcsine" transformation is one which has a faster convergence to normality than the logistic transformation. Set $Y=\frac{2}{\pi}\arcsin{\sqrt{\frac{X}{N}}}$ (where $X$ is number of "successes" and $N$ the number of trials), and you can show with the so called "delta method" that the variance of $Y$ is approximately constant (and independent of $Y$, as it should be in the normal distribution). $\endgroup$ – probabilityislogic Jan 21 '11 at 11:49
  • $\begingroup$ The link you provide for "exact probabilities" is broken. Do you have another one? $\endgroup$ – Stephan Kolassa Nov 11 '15 at 15:54
  • $\begingroup$ @StephanKolassa You can find the Clopper Pearson formulae here as well : en.wikipedia.org/wiki/… $\endgroup$ – Joris Meys Nov 19 '15 at 9:44
24
$\begingroup$

To see why it should not be symmetric, think of the situation where $p=0.9$ and you get 9 successes in 10 trials. Then $\hat{p}=0.9$ and the 95% CI for $p$ is [0.554, 0.997]. The upper limit cannot be greater than 1 obviously, so most of the uncertainty must fall to the left of $\hat{p}$.

|cite|improve this answer
$\endgroup$
9
$\begingroup$

@Joris mentioned the symmetric or "asymptotic" interval, that is most likely the one you are expecting. @Joris also mentioned the "exact" Clopper-Pearson intervals and gave you a reference which looks very nice. There is another confidence interval for proportions which you will likely encounter (note it is also not symmetric), the "Wilson" interval which is a type of asymptotic interval based on inverting the score test. The endpoints of the interval solve (in $p$) the equation $$ (\hat{p} - p)/\sqrt{p(1-p)}=\pm z_{\alpha/2} $$

Anyway, you can get all three in R with the following:

library(Hmisc)
binconf(29, 38, method = "asymptotic")
binconf(29, 38, method = "exact")
binconf(29, 38, method = "wilson")

Note that method "wilson" is the same confidence interval used by prop.test without Yates' continuity correction:

prop.test(29, 38, correct = FALSE)

See here for Laura Thompson's free SPLUS + R manual which accompanies Agresti's Categorical Data Analysis in which these issues are discussed in great detail.

|cite|improve this answer
$\endgroup$
  • 1
    $\begingroup$ (+1) Nice that you cite Laura's textbook and add this complement of information about Wilson's CIs. $\endgroup$ – chl Nov 19 '10 at 13:52
  • 2
    $\begingroup$ Thanks. I would like to point out that the Wilson interval is discussed in the article that @Joris referenced. $\endgroup$ – user1108 Nov 19 '10 at 14:35
9
$\begingroup$

There are symmetric confidence intervals for the Binomial distribution: asymmetry is not forced on us, despite all the reasons already mentioned. The symmetric intervals are usually considered inferior in that

  1. Although they are numerically symmetric, they are not symmetric in probability: that is, their one-tailed coverages differ from each other. This--a necessary consequence of the possible asymmetry of the Binomial distribution--is the crux of the matter.

  2. Often one endpoint has to be unrealistic (less than 0 or greater than 1), as @Rob Hyndman points out.

Having said that, I suspect that numerically symmetric CIs might have some good properties, such as tending to be shorter than the probabilistically symmetric ones in some circumstances.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ With regard to the last sentence: then why not calculate the shortest confidence interval (which has equal density values instead of equal interval width or equal tail area to both sides)? With regard to 2.: having the same width to both sides of $\hat p = k/n$ does not imply that a (the normal) approximation must be used. I'd say that this particular interval does not exist if the limits would need to be extended outside [0, 1]. $\endgroup$ – cbeleites Jul 28 '11 at 7:56
  • $\begingroup$ @cb I don't follow this. First, a shortest CI will not necessarily have equal densities at each end. Second, the comment about "does not exist" makes no sense to me: what does "not exist" mean? $\endgroup$ – whuber♦ Jul 29 '11 at 12:49
  • 1
    $\begingroup$ shortest CI. To calculate the shortest CI for a given coverage, I'd start at the max density and enlarge a short step to the side where density is higher. There I get most confidence coverage (for the short step that is). I enlarge the c.i. repeatedly until I have the desired area (coverage). If my steps are small (infinitesimal) the density at both sides will be (approx.) the same. Did I make a mistake in this strategy? $\endgroup$ – cbeleites Jul 31 '11 at 18:01
  • $\begingroup$ does not exist: e.g. 4 successes out of 5. It does make sense to ask for the 95 % c.i. However if I calculate the probability density for the true $p$ given that I observed 4 successes out of 5 trials, the tail above $\hat p = 4/5 = 0.8$ is only about 0.35. Thus instead of accepting e.g. the normal approximation saying the 95% c.i. goes up to 1.15 (which cannot be correct as the the true $p$ of the binomial trial cannot exceed 1, I'd say the c.i. with equal width towards lower and higher $p$ does only exist for confidence levels $< 70 \%$. $\endgroup$ – cbeleites Jul 31 '11 at 18:20
  • 1
    $\begingroup$ Are we talking about different things? The binomial distribution is discrete, a c.i. would be "for $p = 0.8$, in 94 % of the repetitions we observe $k \in \{3, 4, 5\}$ successes in $n = 5$ tests". But I understood that we are to estimate $p$ for already observed $n$ and $k$. E.g. $p$ given that $k = 4$ out of $n = 5$ tests were successes. So I'm talking about $Pr (p | n = 5, k = 4)$, $p \in [0, 1]$. This is not the binomial distribution $Pr (k | n, p)$ but that of proportion $p$ (I don't know its name). Please help me to understand why there is no density for this distribution? $\endgroup$ – cbeleites Aug 1 '11 at 7:23
6
$\begingroup$

Binomial distribution is just not symmetric, yet this fact emerges especially for $p$ near $0$ or $1$ and for small $n$; most people use it for $p\approx 0.5$ and so the confusion.

|cite|improve this answer
$\endgroup$
2
$\begingroup$

I know that it has been a while, but I thought that I would chime in here. Given n and p, it is simple to compute the probability of a particular number of successes directly using the binomial distribution. One can then examine the distribution to see that it is not symmetric. It will approach symmetry for large np and large n(1-p).

One can accumulate the probabilities in the tails to compute a particular CI. Given the discrete nature of the distribution, finding a particular probability in a tail (e.g., 2.5% for a 95% CI) will require interpolation between the number of successes. With this method, one can compute CIs directly without approximation (other than the required interpolation).

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.