Is there a guide, tradition, or accepted practice on what to take into account and in what sequence between VIF, interaction effects, variable transformations and polynomial associations when performing feature selection for multiple linear regression models?
$\begingroup$
$\endgroup$
7
-
4$\begingroup$ What makes feature selection a desirable goal? How do you corrrect standard errors to account for large biases in them caused by feature selection? $\endgroup$– Frank HarrellCommented Mar 26, 2021 at 11:27
-
$\begingroup$ I'm still building my knowledge base in data science and want to understand this aspect of machine learning in all its strengths and weaknesses. Which course other than feature selection would you advise here Frank? Enlighten me. $\endgroup$– rahul-ahujaCommented Mar 26, 2021 at 12:24
-
$\begingroup$ Also inform me, of the answer to my question, disregarding the limitations of that approach for the moment. $\endgroup$– rahul-ahujaCommented Mar 26, 2021 at 12:26
-
$\begingroup$ He's written a book, Regression Modelling Strategies, that discusses this issue extensively. You can also read through some of our existing threads categorized under the feature-selection tag. This issue has been discussed extensively on the site many times. $\endgroup$– gung - Reinstate MonicaCommented Mar 26, 2021 at 18:52
-
2$\begingroup$ Feature selection is usually doomed to not find the right features unless you have orthogonal features and an enormous sample size. Instead concentrate on penalization (shrinkage) and data reduction (unsupervised learning). The latter will reduce the predictor space down to what is stable. Sparse principal components analysis is one of the recommended data reduction approaches. $\endgroup$– Frank HarrellCommented Mar 28, 2021 at 12:58
|
Show 2 more comments