Can I get the score of importance for each feature in feature selection methos such as Chi2, Information Gain (IG), or Recursive Feature Elimination (RFE)? Or they just provide a list of important features?


As far as I know, in scikit-learn you can only get individual (relative) features importance if you're using an ensemble method of decision trees. After fitting a model you can check model.feature_importances_. As in the example below:

# Feature Importance
from sklearn import datasets
from sklearn import metrics
from sklearn.ensemble import ExtraTreesClassifier
# load the iris datasets
dataset = datasets.load_iris()
# fit an Extra Trees model to the data
model = ExtraTreesClassifier()
model.fit(dataset.data, dataset.target)
# display the relative importance of each attribute

Source: https://machinelearningmastery.com/feature-selection-in-python-with-scikit-learn/

This question has also some more in-depth answers on the subject.

| cite | improve this answer | |
  • $\begingroup$ I don't think you are answering his question here, he is asking about specific feature importance algorithms not just if any exist. $\endgroup$ – astel Mar 11 '19 at 19:37
  • $\begingroup$ @astel a matter of adverb. Fixed, thanks for speaking out. $\endgroup$ – Lucas Farias Mar 11 '19 at 19:41

I don't use scikit-learn so I can't speak to the specifics on that, but the R package CORElearn will give you importance values (usually on [0,1] I think) for features using information gain, minimum description length, and a whole host of others. Might be of interest to check that package out if scikit-learn can't provide what you are seeking.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.