1
$\begingroup$

I'm looking the snippet below from ESL. I'm a hard time deriving the variance term (last term in Eq. 7.12).

I started with \begin{align} \frac{1}{N}\sum_i ||h(x_i)||^2\sigma^2 \\ = \frac{1}{N}\sigma^2\sum_i ||h(x_i)||^2 \\ = \frac{1}{N} \sigma^2 \sum_i x_i^T(X^TX)^{-1}x_i\\ \end{align}

If I compare my last expression and their's it would seem the summation is equal to $p$. Assuming my above steps are correct, I can't see how this simplifies to $p$.


enter image description here

$\endgroup$
1
$\begingroup$

The key step is to note that the sum is the trace of the hat matrix, i.e. $$ \sum_i x_i^T(X^TX)^{-1}x_i = \text{tr}\left(X(X^TX)^{-1}X^T\right) \\ = \text{tr}((X^TX)^{-1}X^TX) = \text{tr}(I_p) = p. $$

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.