1
$\begingroup$

I am using different feature selection methods for a regression problem in order to rank the features according to their importance. So far I have used scikit-learn methods f_regression and mutual_info_regression. From statistical methods I have used Pearson and Spearman correlation coefficients. In addition, I have implemented RReliefF method for regression targets. All these methods are able to rank features according to weights/scores.

Are there any additional methods which would be suitable for regression problems?

Improve this question
$\endgroup$
2
  • $\begingroup$ The issues of feature selection in regression are covered extensively in the answers to this question and on other threads on this site. Please look over the discussion on that page and edit this question to emphasize the issues that you still think are unresolved. $\endgroup$
    – EdM
    Commented Mar 15, 2019 at 19:13
  • $\begingroup$ Hi, thanks for the link and I read it through. However, I don't try to find out if these feature selection methods are good or not, but rather try to collect all available methods to do feature selection for regression problems. My internal validation metric then compares all these methods in order to evaluate their performance in choosing features. $\endgroup$
    – wieus
    Commented Mar 16, 2019 at 14:53

1 Answer 1

Reset to default
0
$\begingroup$

I know no Python so I don't really know what you're talking about. Is there a possibility to build the feature stepwise?

This means, first you build the model with all the features, then compare that fit with other models identical to the original except one variable is removed. Take the best of those and repeat the process until no variable removal improves the model anymore.

Also, the adjusted R^2 can give you a hint about what features improve/worsen the model (the process above described is usually based on AIC instead, which is a different criteria)

Anyway, the right approach is not to put as many variables as possible and then make feature selection. Instead, try to find beforehand which variables may be more relvant with exploratory analysis (i.e. make some good graphs)

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.