# Estimating probability or frequency with low N?

I am trying to estimate the probability of an event using a low number of observations. The naive estimator

$\hat{p} =\frac{\text{number of positive observations}}{\text{total number of observations}}$

works well when the total number of observations is big enough, but if you have only a few observations, there is a decent chance that you will erroneously conclude to a 0 or 100% probability.

I suppose you could set a prior distribution on the estimated probability (say, uniform), and look for better estimators. I suppose this problem has already been tackled many times, so where should I look?

• This document might be of interest. There is an online calculator that calculates point estimates and confidence intervals based on the different methods discussed in the paper. Jul 12 '13 at 9:32
• With small N, another, but related, problem to consider is that the number of possible estimates is fairly limited. Say N=10, then there are only 11 possible estimates of $\hat{p}$: 0%, 10%, 20% ... 90%, 100%. This is probably trivial, but it is also suprising easy to overlook. Jul 12 '13 at 9:37

Yes, if you do actually have some prior information, the Bayesian approach might be helpful. In which case you might want to consider a Beta distribution for the prior, because this is a conjugate prior when your random variable is binomially distributed, i.e. makes the maths simple by also giving you a Beta distribution for the posterior. So with $X \sim B(n,\theta)$, and a Beta prior, $\theta \sim \mathrm{Beta}(a,b)$, then with $x$ successes in $n$ trials, your posterior is given by $\theta|x \sim \mathrm{Beta}(a + x,b + n -x)$.
The uniform distribution is a special case of the Beta distribution (for the parameters $a=b=1$). However, a uniform prior is uninformative (it's flat, i.e. asserts each possible value of $\theta$ is equally likely), and you don't do any better than the classical approach (in fact I believe you essentially get the same point estimate / credible interval). So if you don't have any prior information, it looks like @COOLSerdash's link in the comment on your question is the way to go to get appropriate point estimates / confidence intervals.
• +1 and a short note for @static_rtti: see here for a complete worked-out example using the $\mathrm{Beta}$ for proportions. Jul 12 '13 at 16:58