Is there a convention when using the Wilson interval and all observations are successes (or failures) to artificially increase or decrease the lower bound of your CI estimate to encompass the observation?

I am calculating a confidence interval for a binomial proportion using the Wilson interval. (For reference, section 3.1 here and section 2.2 here.)

If you observe 100% success, the upper bound of the estimated CI will not be 1, though it approaches it asymptotically as n increases. That makes intuitive sense to me. Yet I see at least one example where the CI is expanded to include 1, despite the calculation. I'm wondering if there is a convention of doing this that I'm missing or if my example tool is an outlier.

A concrete example: I observe 60 successes in 60 trials. Using equation (4) of Brown et al. (first link, above) and K = 1.96 I get a 95% CI of (0.955, 0.985). However, the tool in the third link above returns (0.940, 1).

The tool's 95% CI is also wider. I'm less concerned about an outright math difference. But I am more concerned with that upper bound. I am calculating 95% CIs for quite a few trials of diagnostic tests, many of which show "perfect" results. Thus I'm looking for a solid justification for why my calculation is less optimistic than the clinicians' observations. Or, if there is a convention justifying post-hoc adjustments, I'd do that.

  • $\begingroup$ Can you edit to include the details of the input and output. When I follow the link I do not see that. $\endgroup$
    – mdewey
    Jun 3 '20 at 12:34
  • $\begingroup$ I've added more information on the specific calculation I'm doing as well as output from the tool. I've also separately emailed the tool authors. $\endgroup$ Jun 3 '20 at 12:50

Philosophical issues arise with binomial CIs when there are no successes or all successes (and for Poisson with no successes). Are you really prepared to contemplate that the the true success probability could be $0$ or $1$?

Examples: (1) You want to know whether Montana is completely free of a cattle disease. You test 1000 randomly chosen cattle and find all disease free. But there are many more than 1000 cattle in Montana. Maybe you wouldn't want to give a CI that includes $0.$

(2) As far as is known, particle G always decays within several microseconds of creation in a high energy collision of particles. Theorists hold out hope for a more stable G particle. Your group has made 1000 G particles all of which decayed almost instantly. What would you mean by a CI that includes $1?$

I suppose there may be 'conventions' about including $0$ or $1,$ or not, depending on the purpose and setting of the CI.

If you are in a setting where a Bayesian approach is appropriate, you could choose a prior distribution that envisions possible $1$ parameter values (or $0$) or a prior that does not. Then your interval estimate would be based on a Bayesian posterior distribution.

The fundamentally Bayesian Jeffreys interval estimate based on an 'uninformative' prior is often used as a frequentist CI (see Wikipedia on binomial CIs). Maybe you should consider it in its 'native' Bayesian sense.

  • $\begingroup$ Thanks for this. My question is specifically around the conventions though, particularly in the biomedical context. What I'm trying to avoid is a situation where my calculation is (rightly or wrongly) considered "outside of the norm" and ignored. $\endgroup$ Jun 3 '20 at 17:15
  • 1
    $\begingroup$ I have seen various conventions. Suggest you look at articles on related topics, ask colleagues, and think about what message you want to give. If you're writing an article for publication you may get more advice than you want from reviewers and editors.// If no clear guidance or opinion, then choose a type of CI, say what it is, and use exactly the result it provides. $\endgroup$
    – BruceET
    Jun 3 '20 at 18:34
  • $\begingroup$ Thanks. I dug around a bit this afternoon and managed to track down the discrepancy. So marking your answer as correct. $\endgroup$ Jun 3 '20 at 19:40

Contrary to your assertion in the question, the Wilson score interval with no successes (failures) should be an interval with an endpoint exactly at zero (one). The form of the interval in this case is as follows. Let $\chi_{1, \alpha}^2$ denote the critical point of the chi-squared distribution with one degree-of-freedom, using the upper-tail area $\alpha$. With no successes (failures), the Wilson score interval reduces to the respective forms:

$$\text{CI}_p(1-\alpha) = \Bigg[ 0 , \frac{\chi_{1, \alpha}^2}{n + \chi_{1, \alpha}^2} \Bigg] \quad \quad \quad \text{CI}_p(0.95) = \Bigg[ 1-\frac{\chi_{1, \alpha}^2}{n + \chi_{1, \alpha}^2}, 1 \Bigg].$$

Using your data with $n=60$ data points and no failures, and taking $\alpha = 0.05$ for a 95% confidence interval, gives you the interval:

$$\text{CI}_p(0.95) = \Bigg[ 0.9398281, 1 \Bigg].$$

As you can see, this interval encompasses the point $\theta = 1$, just as it should. Consequently, there is no need to artificially alter the bounds of the interval. If you would like to compute the Wilson-score interval, this has been implemented in the CONF.prop function in the stat.extend package. Applying that function for your data confirms the above manual calculations.

#Compute the confidence interval (with no failures)
CONF.prop(sample.prop = 1, n = 60, alpha = 0.05)

        Confidence Interval (CI) 
95.00% CI for proportion parameter for infinite population 
Interval uses 60 binary data points with sample proportion = 1.0000 

[0.93982814785791, 1]

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.