$Y_{1...n}\sim \operatorname{Bin}(1,p)$, iid, and I need to find an unbiased estimator for $\theta=\operatorname{var}(y_i)$.

I did some calculations and I think that the answer is $p(1-p)-\frac{p(1-p)}{n}$

  • Is this correct?
  • If not, how can I find an unbiased estimator?
  • $\begingroup$ If $p=\frac{\sum{y}}{n}$, it's not unbiased, as you can check by working out its expectation. Why don't you show your calculations & perhaps someone will point out the error. $\endgroup$ – Scortchi - Reinstate Monica Jun 15 '13 at 13:26

This answer cannot be correct. An estimator cannot depend on the values of the parameters: since they are unknown it would mean that you cannot compute the estimate.

An unbiased estimator of the variance for every distribution (with finite second moment) is

$$ S^2 = \frac{1}{n-1}\sum_{i=1}^n (y_i - \bar{y})^2.$$

By expanding the square and using the definition of the average $\bar{y}$, you can see that

$$ S^2 = \frac{1}{n} \sum_{i=1}^n y_i^2 - \frac{2}{n(n-1)}\sum_{i\neq j}y_iy_j,$$

so if the variables are IID,

$$E(S^2) = \frac{1}{n} nE(y_j^2) - \frac{2}{n(n-1)} \frac{n(n-1)}{2} E(y_j)^2. $$

As you see we do not need the hypothesis that the variables have a binomial distribution (except implicitly in the fact that the variance exists) in order to derive this estimator.

| cite | improve this answer | |
  • 1
    $\begingroup$ I think the OP is distinguishing between (small) $p$ the statistic $\frac{\sum{y}}{n}$ & (big) $P$ the binomial parameter, though perhaps not. $\endgroup$ – Scortchi - Reinstate Monica Jun 15 '13 at 17:40
  • $\begingroup$ Ah, I did not think about this. Is it something standard? $\endgroup$ – gui11aume Jun 15 '13 at 18:53
  • $\begingroup$ No, just my guess. More standard would be to use Greek letters for parameters, capital Latin letters for random variables & small Latin letters for their observed outcomes. In any case, $p$ was introduced without definition. $\endgroup$ – Scortchi - Reinstate Monica Jun 15 '13 at 19:00

@gui11aume is right of course. An outline of a derivation specific to a $\operatorname{Bin}(1,\pi)$ distribution follows:

  1. Find the variance in terms of $\pi$ to reparameterize the probability mass function: $$\theta=\operatorname{Var}{Y_i}=\pi(1-\pi)$$
  2. Find the maximum-likelihood estimator of $\theta$: $$\hat\theta=\frac{\sum{y_i}}{n}\left(1-\frac{\sum{y_i}}{n}\right)$$
  3. Calculate its expectation: $$\newcommand{\E}{\operatorname{E}}\E\hat\theta=\theta\cdot\frac{n-1}{n}.$$ Note thankfully that the bias term is a constant.
  4. Write the unbiased estimator: $$\tilde\theta=\frac{\hat\theta}{\frac{n-1}{n}}=\frac{\sum{y_i}}{n}\left(1-\frac{\sum{y_i}}{n}\right)\cdot\frac{n}{n-1}=p(1-p)\cdot\frac{n}{n-1}$$ where $p$ is the statistic $\frac{\sum{y_i}}{n}$

Because $\sum{y}$ is sufficient & complete, $\tilde\theta$ is not just any unbiased estimator of the population variance, but the unique minimum-variance unbiased estimator.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.