# Why is the expected value of the optimism of the training error for a linear model equal to $\frac{2}{ n}d\sigma ^{2}$?

In the book Elements of statistical learning 2 on page 229, they express the expected optimism of the training error as: $$\omega=\frac{2}{N} \sum_{i=1}^{N} \operatorname{Cov}\left(\hat{y}_{i}, y_{i}\right)$$

This is followed by the following paragraph:

This expression simplifies if $$\hat{y}_{i}$$ is obtained by a linear fit with $$d$$ inputs or basis functions. For example, $$\sum_{i=1}^{N} \operatorname{Cov}\left(\hat{y}_{i}, y_{i}\right)=d \sigma_{\varepsilon}^{2}$$ for the additive error model $$Y=f(X)+\varepsilon$$

I'm struggling to see where this relation comes from. Why is the sum of the covariances between predicted and true values equal to the the number of basis functions times the variance of the errors in the linear model? What are basis functions in this context, exactly?

Since we are in the case of a linear fit of the additive error model $$\hat{{y}} = X(X^TX)^{-1}X^Ty$$ And we notice that $$\sum_{i=1}^{N}{cov(\hat{y}_i, y_i)}$$ is just the sum of the diagonal elements of the covariance matrix $$\Sigma = cov(\hat{y}, y)$$. In other words
$$\sum_{i=1}^{N}{cov(\hat{y}_i, y_i)} = trace(\Sigma)$$
Putting these facts together we obtain \begin{aligned} trace(\Sigma) &= trace(cov(\mathbf{\hat{y}}, \mathbf{y})) \\ &= trace(cov(\mathbf{X}(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T \mathbf{y}, \mathbf{y})) \\ &= trace(\mathbf{X}(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^Tcov(\mathbf{y}, \mathbf{y})) \\ &= trace(\mathbf{X}(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T)\sigma^2_\epsilon \\ &= trace((\mathbf{X}^T\mathbf{X})^{-1}(\mathbf{X}^T\mathbf{X}))\sigma^2_\epsilon \ \ \ \ \ \ \ \ \text{ (permuting inside the trace)} \\ &= trace(\mathbf{I}_d)\sigma^2_\epsilon \\ &= d\sigma^2_\epsilon \ \ \ \ \ \ \ \ \text{ (as required)} \\ \end{aligned}