1
$\begingroup$

My professor proposed the below relationship as a property of the variance (he called $E[(X - c)^2]$ mean squared error):

$$ E[(X - c)^2] = Var(X) + (E[X] - c)^2 $$

and he said that, when $c = E[X]$, the mean squared error is minimized.

I'm wondering which is the name of this property/relationship.

$\endgroup$
1
  • 1
    $\begingroup$ This is the statistics version of Pythagoras' Theorem. $\endgroup$
    – Xi'an
    Commented Feb 17, 2022 at 17:01

1 Answer 1

2
$\begingroup$

The identity you have is simply the bias-variance decomposition of the mean squared error: $$ E[(X - c)^2] = Var(X) + (E[X] - c)^2\quad (1), $$

and it is proven quite simply by rearranging the identity for the variance $$Var(Y)=E[Y^2]-(E[Y])^2$$

applied to the case $Y=X-c$.

Take $X$ as given and $c$ as a choice argument. Then it is clear from the right hand side of $(1)$ that since the variance term is independent of $c$ and the second term (bias squared) is nonnegative, the whole expression is minimized if you choose $c=E[X]$ (the bias term is then zero).

If you are interested in Bayes estimation, this is essentially the reasoning behind why any Bayes estimator under quadratic loss is just the posterior mean: it minimizes the posterior expected loss.

$\endgroup$
2
  • $\begingroup$ Hello @Golden_Ratio thank you very much for your helpful answer. $\endgroup$ Commented Feb 17, 2022 at 11:42
  • $\begingroup$ @GennaroArguzzi no prob! $\endgroup$ Commented Feb 17, 2022 at 11:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.