What sample size $n$ is required to test the following binomial probability in a single sample:
$H_0: p \leq p_0$
$H_1: p > p_0$
Is there an exact, robust formula to calculate $n$ in terms of $\alpha$, $\beta$, expected probability $p_0$, and effect size (ideally Odds Ratio), when $p_0$ is very small (e.g., <10^-12)?
Let $X \sim \mathcal{Binom}(n,p)$. Under the assumption that the value of $p$ is the very small $p_0$ (as an example I use $p_0 = 10^{-6}$). We will not be able to do a meaningful test unless $n$ is very large, so if needed $X\sim \mathcal{Poisson}(n p_0)$ will be a very good approximation. The waiting time to thhe first success will be a geometrical variable $N$ with pmf (probability mass function) $(1-p)^{n-1}\cdot p$ with expected value $1/p$, so with our $p_0$ that is $10^6$, so we must plan for a rather large sample!
But even then there is a risk (if our assumption is true) that we will not see any success, so we must plan for that. Then remember the rule of three Revisiting the Rule of Three, if we did not see any success then an approximate confidence interval (CI) is $[0,1/3n]$. If you want such an interval to be no larger that $[0,p_0]$ you must plan for $1/3n = p_0$, that is, a sample size of about $333 333$.
This is admittedly only a back-of-the-envelope calculation, I will add some formal power calculations later.
