Detection Fraction from Counting Experiment: Poisson or Binomial Uncertainty?

Question

Suppose I have 100 objects. I have some "detection criteria" such that I can tell you that 25/100 objects have been "detected." Therefore, my detection fraction is 25%. Now, I want to derive an error bar for my detection experiment. I can do this easily if I assume a binomial distribution: $df = \sqrt{f(1-f)/N} = \sqrt{0.25\times0.75 / 100} \approx 0.043 = 4.3\%$.

On the other hand, someone might ask: well, why did you assume the binomial distribution instead of the Poisson distribution? You've underestimated your errors! Your actual error in the detection fraction is actually $\sqrt{N_{det}}/N_{total} = \sqrt{25}/100 = 0.05 = 5\%$.

So, which error bar is more correct: Binomial or Poisson? And is using the Binomial distribution really misleading?

Note that for the "low detection fraction" regime, Binomial uncertainty does indeed start to become significantly lower than the Poisson uncertainty. For example, with a detection fraction of $9/25=0.36$, the Poisson uncertainty is $12\%$ whereas the Binomial uncertainty is $9.6\%$. This stuff then becomes important for assessing whether your detection fraction is "statistically significant" -- e.g., whether $f/df\geq3$ (a so-called $3\sigma$ result).

Answer 1

The two seem to be based on different experimental setups, or at least in terms of what parameters are considered known vs. uncertain.

Your Binomial approach assumes the number of objects $N=100$ is known with no uncertainty. Then it is assuming the number of detected objects $n$ is Binomial distributed, $n|N\sim\mathrm{B}_{N,p}$, and making inferences about the unknown detection probability $p$.

The Poisson distribution is characterized by the average occurrence rate of events in time $\lambda=\mathbb{E}\big[N|T\big]$, where the time interval $T$ is known. (This would be a length, if the events are distributed in space rather than time.) So in the Poisson model, the number of events is unknown, $N\sim\mathrm{P}_\lambda$. In this case*, given the above Binomial detection model for $n|N$, then $n\sim\mathrm{P}_{\lambda p}$ is the marginal distribution of $n$. That is, your Poisson approach appears to be using a marginal distribution for $n$, which integrates over all possible values of $N$, i.e. assuming $\lambda$ is fixed, but $N$ is unknown. This likely accounts for the greater uncertainty. (*Assuming I have interpreted the Wikipedia link correctly!)

If $N$ is truly known, then $\lambda$ is simply a nuisance parameter, because even if object occurrence is Poisson, this has no bearing on $p$. Therefore the Binomial approach would be more appropriate. (Note however that your version is not the only possible one for the Binomial case.)