0
$\begingroup$

I'm currently reading James1997 - Generalizations of the Bias/Variance Decomposition for Prediction error.

My ultimate goal is to see how, in the special case of squared loss, bias-effect and variance-effect reduce to the commonly known bias (squared) and variance.

There is an arithmetic trick being used that I somehow cannot figure out. Let $Y$ and $\hat{Y}$ be random variables. The claim is that

$$ \mathbb{E}_{}\left[ (Y-\hat{Y})^2 \right] = \mathbb{E}_{}\left[ (Y-\mathbb{E}_{}\left[ Y \right] )^2 \right] + \mathbb{E}_{}\left[ (\hat{Y} - \mathbb{E}_{}\left[ Y \right] )^2 \right] $$

Why is that? I've tried calculating but the cross terms do not seem to work out.

Improve this question
$\endgroup$
4
  • $\begingroup$ Welcome to cv, ngmir! The formula that you are trying to show is not correct in general. Where in the paper did you fond it? $\endgroup$
    – Ute
    Jul 26 at 12:22
  • $\begingroup$ Hey, it might well be I misunderstood something. The formula in question is given in the paper in sec. 1.1, just after "However, from the decomposition...". As far as I understood, for squared loss, $SY = E[Y]$ and $S \hat Y = E[\hat Y]$. A similar trick (with different signs) is also applied on page 6 for $bias^2$. $\endgroup$
    – ngmir
    Jul 26 at 12:36
  • $\begingroup$ Your expression is dubious for many reasons, including the point that the purpose of the model is to reduce the expected mean square error of the fitted values as far as possible, and certainly below the variance of the dependent variable (since that can be achieved by guessing the mean) $\endgroup$
    – Henry
    Jul 26 at 13:51
  • $\begingroup$ Okay, the trick I was looking for is actually the highlighted part in this question (thanks for linking!), an alternative approach can be found here $\endgroup$
    – ngmir
    Jul 26 at 16:44

0

Reset to default

Browse other questions tagged or ask your own question.