# Estimation of the variance of MLE with small sample size and binomial distribution

Let $x$ be an observation from $X\sim Bin(n,p)$. I want to estimate $p$ and use ML estimator, $\widehat{p}=\frac{x}{n}$. I also want to estimate the variance of the estimator $\widehat{p}$. It equals: $Var(\widehat{p})=Var(\frac{X}{n})=\frac{Var(X)}{n^2}=\frac{p(1-p)}{n}$. As we don't know $p$, it is often replaced by $\widehat{p}$ in the formula. We obtain: $$Var(\widehat{p})=\frac{\widehat{p}(1-\widehat{p})}{n}.$$ In my case, as $x=0$, $\widehat{p}=0$ and thus $Var(\widehat{p})=0$. We thus see that by approximating $p$ with $\widehat{p}$, we obtain a null variance. I will explain now why it's wrongly estimated. The reason why I get $x=0$ is because $p$ is low (but not null) and because the sample size $n$ is low. But when estimating $Var(\widehat{p})$, we approximate $p$ with $\widehat{p}=0$, which gives us a null variance.

How can we correct for that?

• If you are confident that "$p$ should be low" (but not zero), a Bayesian approach suggests itself in which you assign a prior probability distribution to $p$ which reflects this knowledge. The resulting posterior variance will then be different from zero. – Christoph Hanck Jan 23 '18 at 13:24
• What kind of prior would you use? – Anthony Jan 23 '18 at 14:07
• Probably a beta prior. – Christoph Hanck Jan 23 '18 at 14:08
• – kjetil b halvorsen Dec 29 '18 at 17:55

Why do you need an estimator of the variance $$np(1-p)$$? Recommendations would probably depend on context. An easier problem is to construct a confidence interval, see Confidence interval around binomial estimate of 0 or 1. A simple ad-hoc solution to get a variance estimator when $$X=0$$ is to take the upper limit of that confidence interval, and base the variance estimate on that.
To find an unbiased variance estimate which is non-zero when $$X=0$$ is probably hopeless (unless it is sometimes negative!) Proving that would be an interesting exercise.