How $\hat{y}_0=x_0^T\beta+\sum^N_{i=1}l_i(x_0)\epsilon_i$? I am currently reading "Elements of Statistical Learning II" and I am not quite sure about one thing in section 2.5 on p.24 and p.26
So at the end of p.24 they write the following:
Suppose that we know that the relationship between $Y$ and $X$ is linear,
$$Y=X^T\beta + \epsilon$$
where $\epsilon \sim N(0,\sigma^2)$ and we fit the model by least squares to the training data. For an arbitrary test point $x_0$, we have $\hat{y}_0=x_0^T\hat{\beta}$, which can be written as $\hat{y}_0=x_0^T\beta+\sum^N_{i=1}l_i(x_0)\epsilon_i$, where $l_i(x_0)$ is the $i$th elements of $X(X^TX)^{-1}x_0$.
I'm struggling to derive how did they arrive to this equation. I know that $\hat{\beta}=(X^TX)^{-1}X^TY$ and so $x_0^T\hat{\beta}=x_0^T(X^TX)^{-1}X^TY$
Since $Y=X^T\beta+\epsilon$ then $x_0^T\hat{\beta}=x_0^T(X^TX)^{-1}X^TY=x_0^T(X^TX)^{-1}X(X^T\beta+\epsilon)$
Well, i'm not sure how exactly to arrive to the equation they do. Can someone show me how?
 A: Let's start with $\underset{N\times 1}{y}=\underset{N\times d}{X}.\underset{d\times 1}{\beta}+\underset{N\times 1}{\epsilon}$, note the dimension of the matrices / vectors underneath.
Hence,
$\sum\limits^N_{i=1}l_i(x_0)\epsilon_i$
$=\langle\;\underset{N\times d}{X}\underset{d \times d \quad}{(X^TX)^{-1}}\underset{d\times 1}{x_0}\quad,\quad \underset{N\times 1}{\epsilon}\;\rangle$, an inner product of two length-$N$ vectors
$=\left(X(X^TX)^{-1}x_0\right)^T.\epsilon$, since $\langle a, b \rangle = a^Tb$
$=\left(x_0^T(X^TX)^{-T}X^T\right).(y-X\beta)$, since $y=X\beta+\epsilon$
$=x_0^T(X^TX)^{-1}X^Ty-x_0^T(X^TX)^{-1}.(X^TX)\beta$, since $(X^TX)^T=X^TX$
$=x_0^T\hat{\beta}-x_0^T\beta$, where $\hat{\beta}=(X^TX)^{-1}X^Ty$
$=\hat{y}_0-x_0^T\beta$
A: The model formulation $\mathbf{Y} = \mathbf{X}^\text{T} \boldsymbol{\beta} + \boldsymbol{\varepsilon}$ is an unusual framing of the regression model (usually $\mathbf{X}$ would be the design matrix but here it is the transpose of the design matrix).  With this formulation you have the OLS estimator $\boldsymbol{\hat{\beta}} 
= (\mathbf{X} \mathbf{X}^\text{T})^{-1} (\mathbf{X} \mathbf{Y})$.  You have:
$$\begin{align}
\hat{\mathbf{Y}}
&= \mathbf{X}^\text{T} \boldsymbol{\hat{\beta}} \\[6pt]
&= \mathbf{X}^\text{T} (\mathbf{X} \mathbf{X}^\text{T})^{-1} (\mathbf{X} \mathbf{Y}) \\[6pt]
&= \mathbf{X}^\text{T} (\mathbf{X} \mathbf{X}^\text{T})^{-1} \mathbf{X}[\mathbf{X}^\text{T} \boldsymbol{\beta} + \boldsymbol{\varepsilon}] \\[6pt]
&= \mathbf{X}^\text{T} (\mathbf{X} \mathbf{X}^\text{T})^{-1} \mathbf{X}\mathbf{X}^\text{T} \boldsymbol{\beta} + \mathbf{X}^\text{T} (\mathbf{X} \mathbf{X}^\text{T})^{-1} \mathbf{X} \boldsymbol{\varepsilon} \\[6pt]
&= \mathbf{X}^\text{T} \boldsymbol{\beta} + [\mathbf{X}^\text{T} (\mathbf{X} \mathbf{X}^\text{T})^{-1} \mathbf{X}] \boldsymbol{\varepsilon} \\[6pt]
&= \mathbf{X}^\text{T} \boldsymbol{\beta} + \mathbf{h} \boldsymbol{\varepsilon}, \\[6pt]
\end{align}$$
where $\mathbf{h}$ is the hat matrix.  So you have:
$$\begin{align}
\hat{Y}_i
= \mathbf{X}_i^\text{T} \boldsymbol{\beta} + [\mathbf{h} \boldsymbol{\varepsilon}]_i 
= \mathbf{X}_i^\text{T} \boldsymbol{\beta} + \sum_{j = 1}^n h_{i,j} \varepsilon_j. \\[6pt]
\end{align}$$
where $h_{i,\ell} = [\mathbf{X}^\text{T} (\mathbf{X} \mathbf{X}^\text{T})^{-1} \mathbf{X}]_{i, \ell} = [\mathbf{X}^\text{T} (\mathbf{X} \mathbf{X}^\text{T})^{-1} \mathbf{X}_i]_{\ell} $.  They use different notation than this (the correspondence is $\ell_j(\mathbf{X}_i) = [\mathbf{X}^\text{T} (\mathbf{X} \mathbf{X}^\text{T})^{-1} \mathbf{X}_i]_j$) but it is the same result.
