1
$\begingroup$

The extremely randomized trees classifier (scikitlearn) provides a (multivariate) feature importance measurement Ensemble methods/feature importance evaluation. For each feature, the classifier produces a statistical measurement (and the corresponding standard deviation) for how important the feature was for predicting the target variable. The basic use of this information is to create a "feature ranking" among the features (from high importance value to low).

The question that I have is if I can use this information to make a conclusion that my features are weak or strong for my training data? Has the size of the "importance value" a meaning in itself? For instance, the highest ranking feature has an importance value of 0.0494 with a standard deviation of 0.024. Can I make the conclusion that the features are weak because the "importance values" are very small?

Improve this question
$\endgroup$
3
  • $\begingroup$ I do not think that you can judge the feature as a weak one relative to the other featuers which are ranked lower. Also, In your example, 0.0494 means almost 5% of the samples is correctly classified by this feature !! $\endgroup$
    – soufanom
    Commented Nov 29, 2012 at 3:38
  • 1
    $\begingroup$ @soufanom Thanks for the clarification of the meaning of the statistical "measurement of importance". Could you expand a little more about your first sentence. My understanding is that if a feature is ranked lower (lower feature importance value), it is less important for the classification of the target data. If a feature is ranked higher it is more important for the classification. In that sense, the "feature importance", can be used to compare one feature to the other and to create a ranking list. $\endgroup$
    – andreSmol
    Commented Nov 29, 2012 at 7:43
  • $\begingroup$ In your post, the feature as stated was ranked highest with the stated importance value. In decision trees, if the feature is ranked in a higher node (not necessarily a high score), then, it is of higher importance in this case. Thus, the node level is the actual ranker rather than the score itself. However, the score can be used to compare the features in the tree !! I hope that I have clarified your point !! $\endgroup$
    – soufanom
    Commented Nov 29, 2012 at 11:53

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.