I'm looking at equation 7.9, section 7.3, page 223 in the book "The elements of statistical learning" in which the mean square error of the regression-estimator is decomposed into bias and variance terms. In my opinion the equations are very problematic as they are written. But let's see what I mean by "problematic" here.
We assume the relation $Y=f(X)+\epsilon$, for some "true" function $f$ and $\epsilon$ with some "true" fixed distribution. In the regression problem, we approximate the true $f$ by the regression function $\hat{f}$ which is derived by the std least squares method, based on the realisations of the random $X, Y$.
$\textbf{The problem}$: For an input $X=x_0$, the author defines (pointwise) a square error loss,
$Err(x_0) = \mathbb{E}[ (Y - \hat{f}(x_0) )^2 | X = x_0 ]$
The first discrepancy here (in my opinion) is that for a fixed vector $x_0$, both $\hat{f}(x_0)$ and $f(x_0)$ are both deterministic and the only stochastic term here is the error term (which in principle is unknown!) Then in the same equation (7.9), he defines bias as:
$\textbf{Bias} = \mathbb{E}[ \hat{f}(x_0)] - f(x_0) $. But $\hat{f}(x_0)$ is deterministic as $\hat{f}$ is an already calibrated function! Correct me if I'm wrong but I think I haven't lost my brain yet. Similarly, there is not such a thing as variance of the deterministic, $\hat{f}(x_0)$, $Var(\hat{f}(x_0))$.
At least for me, what is meaningful to define is the following prediction/generalisation error:
$Err = \mathbb{E}[ (Y - \hat{f}(X) )^2 ]$
with bias and variance (of the $\hat{f}(X)$ estimator of the real $f(X)$) defined as:
$\textbf{Bias} = \mathbb{E}[ \hat{f}(X) - f(X) ], \hspace{3mm} Var(\hat{f}(X))$.
Do I say something wrong here or the notation of the author is completely wrong in this equation?? By the way I have the best feelings for the author.