While reading Elements of Statistical Learning, I encountered the following:
Let $$ Y = X^T \beta + \epsilon $$
with $\epsilon \sim \mathcal{N}(0, \sigma^2)$.
Fit an estimator to learn $\beta$ by least squares regression.
For an arbitrary point $x_0$ we have $$ \hat{y}_0 = x_0^T \beta + \sum_{i=1}^N \ell_i(x_0)\epsilon_i $$ where $\ell_i(x_0)$ is the $ith$ element of $$ \mathbf{X} \left( \mathbf{X}^T \mathbf{X} \right)^{-1} x_0 $$
I have read through the book from the beginning and am not sure where this comes from or how to interpret it. I have the basic mathematical background required to understand most of the material but am stumped here. Can someone help me understand
- Where the summation term comes in, and
- What exactly is $\mathbf{X} \left( \mathbf{X}^T \mathbf{X} \right)^{-1} x_0$ and how did it come about?