Inference in binomial with zero successes and fixed number of trials

Let $$X \sim \mathsf{Bin}(n, p)$$ where $$n$$ is known and $$p$$ is to be inferred from the data. Suppose further that $$X = 0,$$ so that we had no successes.

We can reason in the following way. We want to reject any value of $$p$$ that makes the observed data very unlikely. Let's say we want to mistakenly reject a true $$p$$ at most $$100\alpha$$% of the time; in other words, we want to make Type I error with level $$\alpha.$$ For any given $$p,$$ $$\mathbf{P}(X = 0) = (1-p)^n;$$ thus, we reject this $$p$$ if $$(1 - p)^n \leq \alpha.$$ This leads to reject $$p$$ if $$1-\alpha^{\frac{1}{n}} \leq p \leq 1$$. Therefore, the value this way of thinking will not reject are in the interval $$(0, 1-\alpha^{\frac{1}{n}}).$$ Note that when $$n \to \infty,$$ this interval shrinks, which is good (the more observations we have and if we still have zero successes the more certain we are $$p \approx 0$$).

Is this method used in practice?

To compare: I ask this because I was checking some materials and these appeal to asymptotic statistics like the Likelihood Ratio, or Rao's score statistic, or Bayesian methods where they choose either a uniform prior or a beta prior. Using Rao's score with $$n = 25,$$ we would get $$(0, 0.132)$$ at level $$\alpha = 0.05$$ while the Bayesian method with uniform prior will give $$(0.001, 0.132)$$ for a 95% equal-tail posterior interval. The method I described above give $$(0, 0.113),$$ and the interpretation is simple, any value outside this interval makes what we saw to happen only $$5$$% of the time or worse (much worse as $$p$$ grows, actually).

• See Rule of three so in your example of $0$ out of $25$ it suggests an upper figure for the $95\%$ confidence interval of $\frac{3}{25}=0.12$ Commented Jan 10, 2022 at 17:15
• @Henry I didn't refer to this interval as a confidence interval since both ends $0$ and $1 - \alpha^{\frac{1}{n}}$ aren't random. Since both $p$ and $\alpha$ are numbers, the event $0 \leq p \leq 1 - \alpha^{\frac{1}{n}}$ is either the whole sample space or is empty. So I am more confused now that the wikipedia article refers to this interval as a confidence interval. Commented Jan 10, 2022 at 17:20
• If this is a hypothesis test (you say "reject"), you should decide the hypothesis before seeing the data. If you want a Bayesian analysis, observing no successes will lead to results substantially driven by the prior distribution you assume. You asked if an such approach as yours is used in practice and I pointed to a well-known example (though it is really an approximation); it is used when either none or all of the observations are successes and so is random when combined with more conventional approaches when some are successes Commented Jan 10, 2022 at 17:30
• You do use a statistic - the number of successes. In this case that happens to equal 0. If it had equaled some other number, you would have had a different CI. Commented Jan 10, 2022 at 18:00
• Your CI calculation is, effectively, "If $X=0$ then $CI \leftarrow f_1(0, \alpha, n)$ else $CI \leftarrow f_2(X, \alpha, n)$," where $f_1$ is your proposal and $f_2$ is the standard CI calculation. There's nothing wrong with having a CI calculation that has an if-statement in it. Commented Jan 10, 2022 at 18:09

There may be a philosophical issue whether a $$0$$ success probability makes sense. For such a problem, one could use a Bayesian credible interval that gives an upper bound on the success probability $$\theta,$$ based on a sequence of say $$n = 50$$ or $$n = 500$$ failures in a row.

Suppose your prior distribution is that $$\theta$$ is small: $$\theta \sim \mathsf{Beta}(1, 10)$$ and density $$f(\theta) \propto (1-\theta)^9.$$

Then after $$n=50$$ trials with $$x=0$$ successes, the binomial likelihood is $$g(x|\theta) \propto (1-\theta)^{50}.$$

Thus, the posterior density is $$(1-\theta)^9\times (1-\theta)^{50}$$ and the posterior distribution on $$\theta$$ is $$\mathsf{Beta}(1,60),$$ which gives a one-sided 95% posterior interval estimate of $$(0, 0.049).$$ [Using R.]

qbeta(.95, 1, 60)
[1] 0.04870291


Similarly, with $$n = 500$$ consecutive failures, the 95% Bayesian posterior interval estimate would be that $$\theta < 0.0059.$$

qbeta(.95, 1, 510)
[1] 0.005856767


Note: If you approach this problem without any opinion as to the value of $$\theta,$$ then you might use the non-informative Jeffries prior $$\mathsf{Beta}(.5,.5).$$ Then 95% Bayesian credible intervals for $$50$$ and $$500$$ consecutive failures, respectively, results would be as shown in R below (rounded to five places).

round(qbeta(c(.025,.975), .5, 50.5),6)
[1] 0.000010 0.048758
round(qbeta(c(.025,.975), .5, 500.5),6)
[1] 0.000001 0.005009

• The corresponding $\alpha = 0.05$ intervals of the method I mentioned are $(0, 0.058155)$ for $n = 50$ and $(0, 0.00597)$ for $n = 500.$ Very similar intervals (the second one more so), yet I feel to use $(0, 1 - \alpha^{\frac{1}{n}})$ is simpler and the underlying assumptions may be also easier to believe. (There is always the issue of the prior being Beta and so on. And yes, this is not a mathematical point but a philosophical one.) What do you think? Commented Jan 10, 2022 at 19:30
• My note shows that the influence of the prior is not great for sample sizes as large as 50 and (especially) 500. The choice probably depends on one's comfort level with Bayesian or frequentist methods. // I did not intend my Answer to contradict approaches suggested in Comments. For practical purposes results are similar. // The intervals shown in the note are sometimes called Jeffreys confidence intervals and used by frequentists. Commented Jan 10, 2022 at 19:50