# Why not using cross validation for estimating the error of a linear model?

Cross validation (CV) seems to be a simple and useful tool for estimating the out-of-sample error of a linear regression model, even though it is rarely used for this purpose. Why that? Is there a better ways for error estimation? Is it possible to estimate the out-of-sample error by only using the training data?

• You need to state your ultimate goal. For linear models we have a closed-form expression for an unbiased estimate of residual variance if you are honest about the predictor degrees of freedom in its denominator. – Frank Harrell Nov 3 '14 at 16:38
• "honest about the predictor degrees of freedom" ... which can become difficult if the data has (or possibly has) a clustered structure. And yes, depending on what you want/need to measure, other procedures may be more appropriate. – cbeleites supports Monica Nov 3 '14 at 16:45
• Perhaps, but a bit more important is that all the predictors were pre-specified and none was deleted by seeming to be 'unimportant'. Otherwise, d.f. needs to be the 'effective d.f.' as in citeulike.org/user/harrelfe/article/13265069 – Frank Harrell Nov 3 '14 at 17:46
• Given the assumptions that go into the closed-form expression and the potential to run into the multiple comparisons problem before you even choose what to regress on, is it not usually better to rely on cross-validation and a test set to estimate the predictive power of a linear model? – rinspy Jul 31 '17 at 9:48

• In analytial chemistry it is used so often that the measures get their name accordingly: e.g. $RMSE_{CV}$ – cbeleites supports Monica Nov 3 '14 at 16:42