Is cross validation an appropriate technique for variable selection and regression coefficient shrinkage?

A former colleague of mine used 10-fold CV to compare the regression coefficients from the 10 training models. Variables were kicked out if the coefficients changed sign. The average coefficient values for the remaining variables was then used to estimate the coefficients of the final model.

Does this homebrewed methodology sound like a valid technique?

  • 4
    $\begingroup$ You may want to take a look at the elastic net method (see also this detailed handout or Tibshirani and coll JSS paper for one implementation in R). There's also When to use regularization methods for regression? among our related discussions on this site. $\endgroup$ – chl Nov 7 '13 at 17:52
  • $\begingroup$ Great, thank you chl, I agree elastic net seems more appropriate in this case. $\endgroup$ – RobertF Nov 7 '13 at 20:16
  • 3
    $\begingroup$ I asked a very related question earlier this year (see: stats.stackexchange.com/questions/52274/…). The answers from that thread suggest that the best approach is to use the 10-fold CV to choose the size of the regularization penalty (i.e. the C_1,C_2 in the elastic net), and then train your model using this penalty on all the data. $\endgroup$ – Berk U. Nov 7 '13 at 23:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.