0
$\begingroup$

In the derivation of the bias variance decomposition for example on Wikipedia or in this question the following identity is used:

$$E[(E[\hat{f}]-\hat{f})(f-E[\hat{f}])]=E[E[\hat{f}]-\hat{f}](f-E[\hat{f}])$$

with the justification that '$(f-E[\hat{f}])$ is constant'.

I do not understand this.

The expectation $E$ is taken over the distribution $P_X$ of the sample-generating random variable $X$, and both $f$ and $\hat{f}$ are functions of the random variable $X$, hence random variables themselves.

Obviously $E[\hat{f}]$ is constant since it is an expected value, but $f$ is a function of the data: It maps a sample $x$ to its true label $f(x)$, and different samples $x$ have different labels, there is nothing constant about this.

I've read all answers on the bias-variance tradeoff and none explained how $f$ can be 'constant' when it takes different values for different samples.

$\endgroup$

1 Answer 1

1
$\begingroup$

The way I've read it,f is the true function in the population, and therefore fixed since the population is assumed fixed (unlike the sample)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.