Wikipedia's text about the Clopper-Pearson interval for binomial proportions

Question

I'm trying to understand the following text currently (i.e., 2019-09-25) in Wikipedia about the Clopper-Pearson interval:

The Clopper–Pearson interval is an early and very common method for calculating binomial confidence intervals.[8] This is often called an 'exact' method, because it is based on the cumulative probabilities of the binomial distribution (i.e., exactly the correct distribution rather than an approximation). However, in cases where we know the population size, the intervals may not be the smallest possible, because they include impossible proportions: for instance, for a population of size 10, an interval of [0.35, 0.65] would be too large as the true proportion cannot lie between 0.35 and 0.4, or between 0.6 and 0.65.

I do understand that in the given example it would be impossible to get an outcome that would represent a binomial proportion of 0.35 (as this would require 3.5 successes, which is not a possible outcome).

However, I believe the CP-interval is meant to represent the range of underlying probabilities of success (the 'true proportions') that have some minimum probability to produce the observed (integer) outcome. As far as I can see, these 'true proportions' can take values between 0.35 and 0.4, or between 0.6 and 0.65.

Am I seeing this wrong, or is the cited text incorrect?

---

UPDATE

Upon reflection I can see where my confusion originates from. In the context I'm working in, we have $N$ clients that have a $p$ probability that something is happening to them. Our observable is the number of clients that have actually encountered this event.

So, we are not sampling from a vase with green and red balls, and the values that $p$ can take on are fully independent of $N$ (they are definitely not integer multiples of $1/N$). I guess, the cited Wikipedia text refers to the vase case, although the introduction of the page seems to be very much applicable to my situation as well:

In statistics, a binomial proportion confidence interval is a confidence interval for the probability of success calculated from the outcome of a series of success–failure experiments (Bernoulli trials). In other words, a binomial proportion confidence interval is an interval estimate of a success probability p when only the number of experiments n and the number of successes nS are known.

Whereas I assumed Wikipedia was referring to my client group of size $N$ as the population, they were actually referring to the size of the vase.

In that light I can now understand the restrictions on the confidence interval and the discussion on the efficiency of the CP interval. Given the continuous nature of my probability value I don't think any of the restrictions of the CP method apply to my case.

Answer 1

There are multiple concepts blended together into that statement on Wikipedia.

• Let's first tackle the statement "the intervals may not be the smallest possible"
• Then look at the different issue "cases where we know the population size"

Point 1

the intervals may not be the smallest possible

It is correct that the intervals are not efficient (too conservative). But for different reasons.

---

The Clopper-Pearson interval contains those values of $p$ for which an hypothesis test at a certain $\alpha$ level would pass (not reject the null). (this is how confidence intervals work)

The problem with the binomial distribution is in this last part "an hypothesis test at a certain $\alpha$". As the outcome variable is discrete and so will be the p-values associated with those outcomes.

Say you wish to test for a coin flipping proces the hypothesis p=0.5 (with a confidence interval you do this for all values of p) with a level $\alpha=0.01$ and you flip eight times then the observations, if the hypothesis would be true would be distributed as

0     1     2     3     4     5     6     7     8
0.004 0.032 0.109 0.219 0.273 0.219 0.109 0.032 0.004

The values 1 untill 7 would make you not reject the hypothesis at this preset 1% level. But this acceptance (ie. the not rejection) would occur 99.2% of the time and not 99% (if p=0.5 would be true).

So this means that the confidence interval will be conservative. It will be containing the actual parameter value more often than stated. The Clopper-Pearson interval will guarantee that the interval contains the parameter at least $x$ percent of the time, but it will not need to be exactly that percentage.

This relates to the further text on wikipedia

For example, the true coverage rate of a 95% Clopper–Pearson interval may be well above 95%, depending on n and θ. Thus the interval may be wider than it needs to be to achieve 95% confidence

Point 2

cases where we know the population size

This refers to sampling from a population with a (known) fixed size. E.g. sampling with replacement from a vase with $n$ balls, out of which a proportion $p$ has a certain characteristic (say red). Obviously, this proportion needs to be a multiple of $1/n$. There can only be an integer number of red balls in the vase.

Say you have a vase with ten balls and wish to predict how many are red by sampling it. Then it makes no sense to speak of the proportion of number of red balls is between 0.35 and 0.65 (equivalent to 'the number of red balls in the vase is between 3.5 and 6.5').

---

So there are two issues that make the Clopper-Pearson interval not the smallest possible. This may have been inconveniently mixed together on Wikipedia or not spelled out very clearly.

The first point actually only tells that the interval is not efficient; it may have a higher true coverage rate. But there is no solution to make the interval smaller. Except when we use prior knowledge and can use Bayesian intervals.

The second point is a bit weird. It is not a typical situation to sample small populations with replacement. I believe that it may have been used as an argument for the inefficiency of the intervals, while instead the first point was meant/supposed to be used.

---

Written by StackExchangeStrike