2
$\begingroup$

In Hastie & al's book Elements of Statistical Learning, there are two subsections covering insample prediction errors and optimism bias (section 7, p.228-230).

Hastie & al explain that defining the insample prediction error as enter image description here(1),

we can take the expected value of this quantity w.r.t. $Y$ and given $X$ and get enter image description here(2)

with $\bar{err}$ the empirical loss. The empirical counterpart would be

enter image description here(3)

which gives us a framework to derive/compare some model selection criteria, especially Mallow's $C_p$.

I am looking for a book or an article that follows the same logic. Especially, I am interested in finding alternative presentations in which the covariance term appears.

Two underlying motives for my question are 1/ to make sure I get Hastie & al's point right and get as much details on it as I can and 2/ to check whether their logic is commonplace, or whether this framework (esp. formula 2) is their's only.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.