1
$\begingroup$

I'm using the random forest classifier (RandomForestClassifier) from scikit-learn on a dataset of two classes (0 and 1). RandomForestClassifier provides directly the importances of the features through the feature_importances_ attribute.

Is features importance in random forest classification depends on classes(0 or 1) of the samples? If I changes the class of data samples (1 and 0), does it effect the features important or they will remain same or it show the features importance of specific class (say 0 or 1)?

I just want to know that the feature importance that comes in RF Classifier are of class 0 or Class 1? Want to check the expressed gene in treatment sample, So the class of the treatment sample should be always 1 and class of the control sample always 0. Is this correct

$\endgroup$
3
  • $\begingroup$ It's hard to follow what you're asking. Can you edit your post to clarify what you have in mind when you "change the class of data samples" and "effect the feature importance"? $\endgroup$
    – Sycorax
    Jul 1, 2021 at 5:49
  • $\begingroup$ @Sycorax I just want to know that the feature importance that comes in RF Classifier are of class 0 or Class 1. I want to check the expressed gene in treatment sample, So class of the treatment sample is should be always 1 and class of the control sample is always 0. Is this correct ? $\endgroup$ Jul 1, 2021 at 6:10
  • $\begingroup$ You can edit your question to include that clarification. Then it will be eligible for re-opening. $\endgroup$
    – Sycorax
    Jul 1, 2021 at 6:17

1 Answer 1

3
$\begingroup$

No, that's not how feature importance works. The feature importance does not understand the classes as treatments. Feature importance essentially measures how well each feature can be used to construct a split that divides the data into the classes. The feature importance does not describe one class individually.

You can verify this by fitting a random forest, and saving the feature importance, and then comparing them to the feature importance of a model with the reversed class labels. Neglecting random variation, the importance measures will be similar.

I'd recommend reading a high-quality reference on random forest, such as Leo Breiman's papers or the treatment in Hastie et al.'s Elements of Statistical Learning.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.