Is there a guide, tradition, or accepted practice on what to take into account and in what sequence between VIF, interaction effects, variable transformations and polynomial associations when performing feature selection for multiple linear regression models?

  • 4
    $\begingroup$ What makes feature selection a desirable goal? How do you corrrect standard errors to account for large biases in them caused by feature selection? $\endgroup$ Mar 26 '21 at 11:27
  • $\begingroup$ I'm still building my knowledge base in data science and want to understand this aspect of machine learning in all its strengths and weaknesses. Which course other than feature selection would you advise here Frank? Enlighten me. $\endgroup$ Mar 26 '21 at 12:24
  • $\begingroup$ Also inform me, of the answer to my question, disregarding the limitations of that approach for the moment. $\endgroup$ Mar 26 '21 at 12:26
  • $\begingroup$ He's written a book, Regression Modelling Strategies, that discusses this issue extensively. You can also read through some of our existing threads categorized under the feature-selection tag. This issue has been discussed extensively on the site many times. $\endgroup$ Mar 26 '21 at 18:52
  • 2
    $\begingroup$ Feature selection is usually doomed to not find the right features unless you have orthogonal features and an enormous sample size. Instead concentrate on penalization (shrinkage) and data reduction (unsupervised learning). The latter will reduce the predictor space down to what is stable. Sparse principal components analysis is one of the recommended data reduction approaches. $\endgroup$ Mar 28 '21 at 12:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.