I need to know how feature importance in python adds up to 100. I have read other answers in stack overflow but could not get what I needed. Can anyone explain how feature importance in python sums up to 100?

  • $\begingroup$ Would this article help you understanding what is going on? $\endgroup$ – Jan Kukacka Oct 4 '18 at 7:47
  • $\begingroup$ Because you normalize the importance values, so they sum to hundred? $\endgroup$ – GeneX Oct 4 '18 at 7:57
  • $\begingroup$ @JanKukacka.Thanks for the suggestion. I will check it out and come back. $\endgroup$ – aneesh cool Oct 4 '18 at 8:32

The feature importance in Random Forest is based on out-of-bag error, and its value is>=0.

As per the Wikipedia page,

"To measure the importance of the j-th feature after training, the values of the j-th feature are permuted among the training data and the out-of-bag error is again computed on this perturbed data set. The importance score for the j-th feature is computed by averaging the difference in out-of-bag error before and after the permutation over all trees. The score is normalized by the standard deviation of these differences."

So essentially the importance estimates are normalized to add up to 100 for convenience.

  • $\begingroup$ Thanks a lot. So first we build the forest. Then we pass down the out of bag samples and permute each variables. Then we take difference between baseline accuracy and accuracy after the variables perturbed. Then we average the differences for a variable for all the trees and divide by its standard deviation. Am I right? $\endgroup$ – aneesh cool Oct 4 '18 at 11:07
  • $\begingroup$ @AneshMuthiah Yes. You re right. You can find more info here eeecon.uibk.ac.at/~zeileis/papers/Lifestat-2008.pdf $\endgroup$ – prashanth Oct 5 '18 at 7:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.