Derivation of the linear regression in sample error equation I'm reading Elements of Statistical learning Chapter 7. I am slightly confused about one term in the derviation of the in sample error equation (equation 7.12)

Can some explain/show mathematically how the variance term changes to become $\frac{p}{N}\sigma_{\epsilon}^2$ in equation 7.12? I know the $1/N$ is due to the fact that the second equation relates to the traning error, and so is an average loss over the training sample. However, I'm unsure where the $p$ comes from.
 A: I'm pretty sure that this is the hard way, but here it is. First, simplify the following:
$$||\mathbf h(x_i)||^2 = \mathbf h(x_i)^T \mathbf h(x_i) = x_i^T \mathbf (\mathbf X^T\mathbf X)^{-1}x_i$$
We want this term to be summed over all $i$. Let the SVD of $\mathbf X^T\mathbf X$ be $\mathbf U\mathbf S\mathbf U^T$ (since it's symmetric and positive semi-definite, this is basically the diagonalised version of it and $\mathbf U\mathbf U^T=\mathbf I$, columns of $\mathbf U$ are orthonormal). Then,  $$(\mathbf X^T\mathbf X)^{-1}=\mathbf U\mathbf S^{-1}\mathbf U^T=\sum_{j=1}^p\frac{1}{\sigma_j} u_ju_j^T$$, where $\sigma_j$ are the diagonal entries of $\mathbf S$. Substitute it in the sum:
$$\begin{align}\sum_{i=1}^n ||\mathbf h (x_i)||^2&=\sum_{i=1}^nx_i^T\left(\sum_{j=1}^p\frac{1}{\sigma_j} u_ju_j^T \right)x_i=\sum_{j=1}^p\sum_{i=1}^n\frac{1}{\sigma_j} x_i^Tu_j u_j^Tx_i\\&=\sum_{j=1}^p\sum_{i=1}^n\frac{1}{\sigma_j} u_j^Tx_ix_i^Tu_j^T=\sum_{j=1}^p\frac{1}{\sigma_j} u_j^T\left(\sum_{i=1}^n x_ix_i^T\right) u_j\\&=\sum_{j=1}^p\frac{1}{\sigma_j} u_j^T(\mathbf X^T\mathbf X)u_j=\sum_{j=1}^p\frac{1}{\sigma_j} u_j^T\left(\sum_{k=1}^p \sigma_ku_ku_k^T\right) u_j\\&=\sum_{j=1}^p\sum_{k=1}^p \frac{\sigma_k}{\sigma_j}(u_j^Tu_k)(u_k^Tu_j)\end{align}$$
Since $u_k^Tu_j=0$ for $k\neq j$, and $1$ if $k=j$, the inside sum is $1$, which yields the total sum $p$.
