5
$\begingroup$

This question already has an answer here:

In LOOCV, in general, the model has to be fit n times (n being the number of observations). The test error is then estimated as a mean of all the individual test errors.

Hastie and Tibshirani mention in section 5.1.2 of their book that with least squares linear or polynomial regression, the cost of LOOCV reduces to that of a single model fit, given by

$CV_{(n)} = \frac{1}{n}\sum_{i=1}^n (\frac{y_i - \hat{y_i}}{1 - h_i})^2$

where $\hat{y_i}$ is the $i$th fitted value from the original least squares fit, and $h_i$ is the leverage statistic.

  1. Could you please explain how this result can be proved?
  2. Even after getting the estimate for the test error, how is the model chosen?
$\endgroup$

marked as duplicate by mdewey, Juho Kokkala, kjetil b halvorsen, AdamO, Jan Kukacka Jun 8 '18 at 8:39

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.