Can I get the score of importance for each feature in feature selection methos such as Chi2, Information Gain (IG), or Recursive Feature Elimination (RFE)? Or they just provide a list of important features?
As far as I know, in scikit-learn you can only get individual (relative) features importance if you're using an ensemble method of decision trees. After fitting a model you can check
model.feature_importances_. As in the example below:
# Feature Importance from sklearn import datasets from sklearn import metrics from sklearn.ensemble import ExtraTreesClassifier # load the iris datasets dataset = datasets.load_iris() # fit an Extra Trees model to the data model = ExtraTreesClassifier() model.fit(dataset.data, dataset.target) # display the relative importance of each attribute print(model.feature_importances_)
This question has also some more in-depth answers on the subject.
I don't use scikit-learn so I can't speak to the specifics on that, but the R package CORElearn will give you importance values (usually on [0,1] I think) for features using information gain, minimum description length, and a whole host of others. Might be of interest to check that package out if scikit-learn can't provide what you are seeking.