8
$\begingroup$

how can I calculate the variance of p as derived from a binomial distribution? Let's say I flip n coins and get k heads. I can estimate p as k/n, but how can I calculated the variance in that estimate?

I'm interested in this so that I can control for variance in my ratio estimates when I'm comparing between points with different numbers of trials. I'm more sure of the estimate of p when n is greater, so I would like to be able to model how reliable the estimate is.

Thanks in advance!

example:

  • 40/100. The MLE of p would be 0.4, but what is the variance in p?
  • 4/10. The MLE would still be 0.4, but the estimate is less reliable, so there should be more variance in p.
$\endgroup$
12
$\begingroup$

If $X$ is $\text{Binomial}(n, p)$ then MLE of $p$ is $\hat{p} = X/n$.

A binomial variable can be thought of as the sum of $n$ Bernoulli random variables. $X = \sum_{i=1}^n Y_i$ where $Y_i\sim\text{Bernoulli}(p)$.

so we can calculate the variance of the MLE $\hat{p}$ as

$$\begin{align*} \text{Var}[\hat{p}] &= \text{Var}\left[\dfrac{1}{n}\sum_{i=1}^n Y_i\right]\\ &= \dfrac{1}{n^2}\sum_{i=1}^n Var[Y_i]\\ &= \dfrac{1}{n^2}\sum_{i=1}^n p(1-p)\\ &= \dfrac{p(1-p)}{n} \end{align*}$$

So you can see that the variance of the MLE gets smaller for large $n$, and also it is smaller for $p$ close to 0 or 1. In terms of $p$ it is maximized when $p=0.5$.

For some confidence intervals you can check out Binomial Confidence Intervals

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I think the link is similar to what I'm looking for, but I want a value that is equivalent to the variance of p. How can I get that from the confidence interval? $\endgroup$ – Jautis Jan 19 '16 at 20:49
  • $\begingroup$ I edited my original answer to more closely answer your question. $\endgroup$ – bdeonovic Jan 19 '16 at 21:13
  • $\begingroup$ How do you deal that the formula of the variance requires p but you only have an estimation of p? $\endgroup$ – Ramon Martinez Sep 3 '18 at 11:15
  • $\begingroup$ You could consider using a variance stabilizing transform such as $arcsin(\sqrt{\hat{p}})$ and then you get that the variance of the transformed variable is $\tfrac{1}{4n}$ $\endgroup$ – bdeonovic Sep 6 '18 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.