Sufficiency is okay. But I don't really get it why the fact guarantees it has minimal variance? Can anyone explain to me somewhat intuitively?


The idea is based on the properties of conditional expectation. Recall that by the variance decomposition, we have

$$ var \left( E\left[ X_1 | X_2 \right] \right) \leq var(X_1)$$

which basically says that the variance of the regression of $X_1$ on $X_2$ is no larger than the variance of $X_1$. That is intuitive, right? We are using information of another variable for prediction so we cannot do worse.

Now, notice that both the random variables $X_1$ and $E\left[ X_1 |X_2 \right]$ have the same mean $\mu_1$ (recall the law of iterated expectations). If we did not know $\mu_1$ then we would try to estimate it from either one of them. Given, nevertheless that the variable on the LHS has minimum variance we would put more reliance to it. Indeed, we would use it as our best guess for the unknown parameter.

Translating this idea to sufficiency, we see that if we can find an unbiased function of the sufficient statistic, which takes the role of $X_2$, then we can estimate with minimum variance. By definition the sufficient statistic is such that conditional probability of your sample does not depend on the unknown parameter, hence this is a valid estimation procedure.

Of course one might still say that we have just proven that we cannot do better than this. We can do just as well with a different statistic, though. This is a valid critique of course and that's why in the sequence we look if the estimator is unique, which can be assured by the completeness of the family of distributions.

| cite | improve this answer | |
  • $\begingroup$ Thank you for you answer. I got it roughly. now I'm trying to understand more precisely one step by step :) $\endgroup$ – Jason Park Oct 30 '15 at 10:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.