7
$\begingroup$

Can someone please help me understanding why add 2 successes and failures in Agresti-Coull interval instead of 3 or 4 or any arbitrary number?

I did my own analysis here and found out that when for example, add 3 successes, the coverage is even better than adding 2, does this introduce any additional bias or problem? I did notice the curvation changed to something similar to no correction, why is that?

If so, can someone help pointing out some references I can read more about?

Many thanks for your help!

Highlights from my analysis:

$\endgroup$
1
  • $\begingroup$ If you want a reference to read about the Agresti-Coull interval, wouldn't you start with the actual paper by Agresti and Coull? (Agresti, A. and Coull, B.A. (1988), Approximate is better than "exact" for interval estimation of binomial proportions, American Statistician, 52, 119-126.) ... incidentally the paper by Brown et al mentioned in the answer by dietervdf is on Brown's academic webpage $\endgroup$ – Glen_b Feb 23 '17 at 23:08
11
$\begingroup$

The reason why Agresti and Coull chose these "add two successes and two failures" lies in rounding of a Wilson 95% CI. It was not determined by simulation studies.

I had to write a small paper on the topic once, here were my findings (condensed) and an answer to your question.

Pro-tip, R has a package propCIs which has different CI's build in. (Agresti-Coull is the add4ci method)

Background

When $X_i \stackrel{d}{=} \text{Ber}(p)$ and the sample size is $n$ then one would intuitively try the following $1-\alpha$ CI (so called Wald CI): $$\hat p \pm z_{\alpha/2}\sqrt{\frac{\hat p(1-\hat p)}{n}}$$ But as you know this behaves badly.

One of the reason the default CI behaves badly, is the usage of $\hat p$ to find the width of the CI. This results in a very small width when $\hat p$ is close to 1 or 0. The Wilson CI tries to mitigate this issue.

This Wilson CI uses the width under $H_0$ and seeks all $p$ which solve $$\left|\dfrac{\hat p - p}{\sqrt{\frac{p(1-p)}{n}}}\right| < z_{\alpha/2}$$

Working this out results in a quadratic equation wich results in the following crazy formula for the CI: $$\hat p \left(\frac{n}{n+z_{\alpha/2}^2}\right) + \frac{1}{2}\left( \frac{z_{\alpha/2}^2}{n+z_{\alpha/2}^2}\right) \pm z_{\alpha/2} \sqrt{\dfrac{1}{n+z_{\alpha/2}^2}\left[ \hat p (1-\hat p) \left( \dfrac{n}{n+z_{\alpha/2}^2}\right)+\dfrac{1}{2}\left(1-\dfrac{1}{2}\right) \left(\dfrac{z_{\alpha/2}^2}{n+z_{\alpha/2}^2}\right)\right]}.$$

This CI behaves pretty good. See the figure below.

Agresti-Coull

Agresti and Coull looked at the center of the Wilson CI and noticed a simplification if one calculates a 95% CI. $z_{0.025} = 1.96\approx 2$.

Now notice how the center of the Wilson CI was given by: $$\hat p \left( \dfrac{n}{n+z^2_{\alpha/2}}\right) + \dfrac{1}{2}\left( \dfrac{z^2_{\alpha/2}}{n+z^2_{\alpha/2}} \right)$$

When you apply the simplification suggested above you find: $$\tilde p = \hat p \left( \frac{n}{n+4}\right) + \dfrac{1}{2}\left( \dfrac{4}{n+4}\right) = \hat p \left( \frac{n}{n+4}\right) + \dfrac{2}{n+4} = \dfrac{X+2}{n+4}$$ Which explaines the "adding two failures, two success - method". The Agresti-Coull CI is then defined as: $$\tilde p \pm z_{\alpha/2}\sqrt{\dfrac{\tilde p(1-\tilde p)}{\tilde n}}$$

Comparison - Coverage probability

The following picture show the three (as well as Clopper-Pearson) and the coverage probability for simultation of 5000 times. CI

Why not use 3 or even more successes/failures ratio's?

First of all, the derivation of the Agresti-Coull interval makes sense.

I've looked at your graph of the performance of the different methods and you claim the "adding three successes" is better, but I'm not convinced. I would say it is worse since the coverage probability is systematically to large. Meaning that the CI are to large, which makes them to conservative. Using this interval it would be harder to detect a significant result.

The paper L. D. Brown et al. (2001). Interval Estimation for a Binomial Proportion. Statistical Science. contains a very good overview of all the CI's for a binomial distribution.

$\endgroup$
2
  • $\begingroup$ Wow, thanks a lot @diervdf, this is super helpful!! And yes, I agree on your last statement as well, I was only evaluating performance by a singular metric, # of coverage, with the trade off of wider CI. $\endgroup$ – Michael Huang Feb 24 '17 at 1:56
  • 1
    $\begingroup$ I know I found it suprising how wel Agresti-Coull behaves, it was a fun an interesting expericience to work with the different CI's for a binomial distribution. BTW: Feel free to accept my answer if it covers the question $\endgroup$ – dietervdf Feb 25 '17 at 13:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.