# Minimizing Expected Prediction Error

Expected Prediction Error in a linear model is defined as follows:

$E[\sum_i (y^*_i-\hat{y_i})^2] = E[\sum_i (y_i-\hat{y_i})^2] + 2\sum_i Cov(y_i,\hat{y_i})$

where $y_i$ are the training values, and $y^*_i$ are the test values (Proof Efron (1986)).

Now if we consider a linear regression setting, based on the above equality, my question is instead of doing cross validation why don't we simply minimize this expression?

The first term on the right hand side can be approximated by Residual Sum of Squares. The problem is to estimate the second term.

• Efron had a number of publications in 1986... – Richard Hardy May 7 '17 at 8:39

This is possible to do. It's called Mallow's Cp. This criteria gives an unbiased estimate of the test set error of data at data with the same input as the training data. (i.e. the $X$'s are the same, but the $y$'s are different.)
• Okay! In a few days I'll have more time to write a more detailed reply. For now, I think I understand your question better. For instance, you're okay with using a method nonlinear in $y$, like, for instance, lasso. --- Note these cov estimation problems come down to estimating the "degrees of freedom" of the estimate. It became pretty common in the past for papers to publish estimates for the degrees of freedom of their method (see arxiv.org/pdf/0712.0881.pdf , arxiv.org/pdf/1409.5391.pdf , for instance) – user795305 May 7 '17 at 14:23