Expected optimism equals sum of covariances

Question

I am reading section for 7.4 in elements of statistical learning, and I have some questions about some definitions and results in this section, and I would like to check if my understanding is correct.

1. In the definition for generalization error $Err_T=\mathbb{E}_{X^0,Y^0}[L(Y^0,\hat{f}(X^0)|T]$, why do we condition on training set $T$ if $X^0,Y^0$ are simply drawn from the true joint distribution? Is it just because the fit model $\hat{f}$ is dependent on the training set $T$?

2. They define in-sample error as $Err_{in}=\frac{1}{N}\sum_{i=1}^N E_{Y^0}[L(Y_i^0,\hat{f}(x_i))|T]$. Here, I assume $Y_i^0~p_{Y|X}(\cdot|x_i)$ What is the difference between $\mathbb{E}_\mathbf{y}[Err_{in}]$ and $Err_{in}$, as the latter already has an expectation over $Y$?

3. Finally, in the derived result $\omega=\frac{2}{N}\sum_{i=1}^N Cov(\hat{y}_i,y_i)$, how does this covariance even make sense as I am assuming $y_i$ is the training label for data point $i$ in the train set $T$, so it should just be deterministic, no? Unless we are treating the train set as a random variable here?

Answer 1

1. Yes, $\hat f$ is random because $\hat f$ is a function of the training set. The generalisation error is also a function of $T$: a 'good' training set might give you lower generalisation error

2. The book defines $E_{\bf y}$ as the expectation over the outcome variable in the training set, fixing the predictors in the training set. If you're using version 2 of the book, it's after equation 7.20. If you're using version 1 you should look for version 2, which is free now. This section has a footnote Indeed, in the first edition of our book, this section wasn’t sufficiently clear [about what is fixed and what is random]

3. Yes, the training set is treated as random here.