The good news where I am is that researchers are doing less stepwise covariate selection now that I've introduced penalized regression. The bad news is that researchers want to use elastic-net regression for variable selection, then use that covariate choice to build an unpenalized model.

Can someone suggest a reference summarizing why this is a bad idea? I can make arguments as to why I wouldn't want to do this ---

  • these are the maximum-likelihood covariates for regression with a certain penalty term, but are not in any sense optimal for a model without a penalty term;

  • the new model fixes some of the coefficient values chosen by the penalized model -- the ones equal to zero -- and then allows the others to vary;

  • you lose the regularization that the penalized model gives you, so the unpenalized model can still bias coefficients high, and tend to overfit;

  • and, of course, the p values and CIs are off, as there was a whole model-selection process before the final regression model.

But I'd like to be better educated on this, and be better able to help my colleagues.

Or, is this process in fact a reasonable thing to do?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.