Assume n samples from Bernoulli distribution with unknown parameter p,

i. e. $x_{1},....,x_{n}\underset{iid}{\sim}Ber\left(p\right)$

It is known that the confidence interval is given by:

$CI=\hat{p}\pm z_{1-\frac{\alpha}{2}}\cdot\sqrt{\hat{p}\left(1-\hat{p}\right)/n}$

where $\hat{p}=\frac{1}n\sum_{i=1}^{n}x_{i}$

This means that the estimator of the variance being used in this formula is:

$s^2 = \hat{p}\left(1-\hat{p}\right) $

The latter is not trivial for me. I've managed to prove that this estimator is biased (with a finite sample size) but failed to prove consistency.

Since it's not clear (for me) if this estimator is consistent, and it's clearly biased, why should we use it and not the usual unbiased estimator?

Is there a full mathematical reason for this? (A reference could be great also). Maybe I'm failing to understand something else?

  • $\begingroup$ Surely $\hat{p}$ needs to be divided by $n$? $\endgroup$ – Alex R. May 2 '17 at 19:11
  • $\begingroup$ You are right, forgot it. $\endgroup$ – RiskyMaor May 2 '17 at 19:20

$\hat{p}$ converges almost surely to $p$ by SLLN. Since the product of two almost-sure convergent random variable converges almost-surely to the product of their limits, it follows that $s^2$ converges almost surely to $p(1-p)$, which also implies it converges in probability, hence consistency.

| cite | improve this answer | |
  • $\begingroup$ Thank you! Last thing - why should you prefer this estimator (which is biased) over the usual unbiased $\frac{1}{n-1}\sum_{i=1}^{n}\left(x_{i}-\bar{x}\right)^{2}$ estimator? $\endgroup$ – RiskyMaor May 2 '17 at 20:35
  • $\begingroup$ @MaorSH you can show that for Bernoulli $X_i$ $\frac 1n \sum_i(x_i-\bar x)^2 = \bar x (1 - \bar x)$, so this reduces to the usual $\frac 1n$ vs $\frac 1{n-1}$ debate, as discussed here for instance: stats.stackexchange.com/questions/3931/… $\endgroup$ – jld May 2 '17 at 21:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.