In the derivation of the bias variance decomposition for example on Wikipedia or in this question the following identity is used:
$$E[(E[\hat{f}]-\hat{f})(f-E[\hat{f}])]=E[E[\hat{f}]-\hat{f}](f-E[\hat{f}])$$
with the justification that '$(f-E[\hat{f}])$ is constant'.
I do not understand this.
The expectation $E$ is taken over the distribution $P_X$ of the sample-generating random variable $X$, and both $f$ and $\hat{f}$ are functions of the random variable $X$, hence random variables themselves.
Obviously $E[\hat{f}]$ is constant since it is an expected value, but $f$ is a function of the data: It maps a sample $x$ to its true label $f(x)$, and different samples $x$ have different labels, there is nothing constant about this.
I've read all answers on the bias-variance tradeoff and none explained how $f$ can be 'constant' when it takes different values for different samples.
