16
$\begingroup$

how can I calculate the variance of p as derived from a binomial distribution? Let's say I flip n coins and get k heads. I can estimate p as k/n, but how can I calculated the variance in that estimate?

I'm interested in this so that I can control for variance in my ratio estimates when I'm comparing between points with different numbers of trials. I'm more sure of the estimate of p when n is greater, so I would like to be able to model how reliable the estimate is.

Thanks in advance!

example:

  • 40/100. The MLE of p would be 0.4, but what is the variance in p?
  • 4/10. The MLE would still be 0.4, but the estimate is less reliable, so there should be more variance in p.
Improve this question
$\endgroup$

1 Answer 1

Reset to default
24
$\begingroup$

If $X$ is $\text{Binomial}(n, p)$ then MLE of $p$ is $\hat{p} = X/n$.

A binomial variable can be thought of as the sum of $n$ Bernoulli random variables. $X = \sum_{i=1}^n Y_i$ where $Y_i\sim\text{Bernoulli}(p)$.

so we can calculate the variance of the MLE $\hat{p}$ as

$$\begin{align*} \text{Var}[\hat{p}] &= \text{Var}\left[\dfrac{1}{n}\sum_{i=1}^n Y_i\right]\\ &= \dfrac{1}{n^2}\sum_{i=1}^n Var[Y_i]\\ &= \dfrac{1}{n^2}\sum_{i=1}^n p(1-p)\\ &= \dfrac{p(1-p)}{n} \end{align*}$$

So you can see that the variance of the MLE gets smaller for large $n$, and also it is smaller for $p$ close to 0 or 1. In terms of $p$ it is maximized when $p=0.5$.

For some confidence intervals you can check out Binomial Confidence Intervals

Improve this answer
$\endgroup$
4
  • $\begingroup$ I think the link is similar to what I'm looking for, but I want a value that is equivalent to the variance of p. How can I get that from the confidence interval? $\endgroup$
    – Jautis
    Jan 19, 2016 at 20:49
  • $\begingroup$ I edited my original answer to more closely answer your question. $\endgroup$
    – bdeonovic
    Jan 19, 2016 at 21:13
  • $\begingroup$ How do you deal that the formula of the variance requires p but you only have an estimation of p? $\endgroup$
    – Heberto Mayorquin
    Sep 3, 2018 at 11:15
  • 1
    $\begingroup$ You could consider using a variance stabilizing transform such as $arcsin(\sqrt{\hat{p}})$ and then you get that the variance of the transformed variable is $\tfrac{1}{4n}$ $\endgroup$
    – bdeonovic
    Sep 6, 2018 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.