4
$\begingroup$

Let $X_1, X_2, ..., X_n \sim Bernoulli(p)$, where $n > 5$. Let $\hat p = \frac{1}{5} \sum_{i=1}^{5} X_i$ be the estimator of $p$. Is it biased?

My guess is no, since $E[\hat p] = \frac{1}{5} \sum_{i=1}^{5} E[X_i] = \frac{1}{5} \sum_{i=1}^{5} p = \frac{5p}{5} = p$.

However, I find this counterintuitive. If I flip a fair coin 5 times to compute the estimator and happen to get heads ($X_i = 0$) every time, then $\hat p = 0$, but the real $p$ is $0.5$, so I am underestimating the parameter $p$.

$\endgroup$
1
  • 1
    $\begingroup$ No matter what is your estimator, you can always find for it such data that it won't work. $\endgroup$
    – Tim
    Sep 21, 2017 at 8:04

1 Answer 1

8
$\begingroup$

Unbiased does not mean that you estimate the parameter correctly in each and every sample, it means that the errors you make in any particular samples tend to average eachother away across many independent samples.

In your case, every sample in which you flip no heads is matched by an equally likely sample where you flip no tails. In the long run you would get about as many no heads samples as no tails, so the number of heads across all such samples averages to 2.5, and the proportion of heads in such samples averages out to $\frac{2.5}{5} = 0.5$

This is then an example of an unbiased but inconsistent estimator.

It's consistent. Your estimator $\hat p$ is a sample mean, and you can calculate the variance of a sample mean quite easily (as long as the flips are independent)

$$ var(\hat p) = \frac{5p(1 - p)}{n} $$

As your sample get's larger, $n \rightarrow \infty$, and so $var(\hat p) \rightarrow 0$. Using Chebyshev's inequality immediately implies that the estimator is consistent.

$\endgroup$
6
  • $\begingroup$ Nice. I had the wrong intuition. This is then an example of an unbiased but inconsistent estimator. $\endgroup$
    – Amanjo
    Sep 21, 2017 at 21:51
  • $\begingroup$ It's also a consistent estimator. Added a discussion to the answer. $\endgroup$ Sep 21, 2017 at 22:07
  • 2
    $\begingroup$ Oops. My variance is wrong: $var(\hat p) = var(\frac{1}{5} \sum_{i=1}^{5} X_i) = \frac{1}{5^2} \sum_{i=1}^{5} var(X_i) = \frac{1}{5^2} \sum_{i=1}^{5} p(1-p) = \frac{p(1-p)}{5}$. Why? $\endgroup$
    – Amanjo
    Sep 22, 2017 at 18:51
  • $\begingroup$ Is it because I should consider all of the $n$ trials? @MatthewDrury $\endgroup$
    – Amanjo
    Sep 23, 2017 at 22:15
  • 2
    $\begingroup$ I'm also unsure why @Amanjo's variance is wrong there. do you have a slightly more explicit derivation you could share? $\endgroup$
    – postylem
    Sep 20, 2020 at 15:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.