I need to know how feature importance in python adds up to 100. I have read other answers in stack overflow but could not get what I needed. Can anyone explain how feature importance in python sums up to 100?
3
-
$\begingroup$ Would this article help you understanding what is going on? $\endgroup$– Jan KukackaCommented Oct 4, 2018 at 7:47
-
$\begingroup$ Because you normalize the importance values, so they sum to hundred? $\endgroup$– FenilCommented Oct 4, 2018 at 7:57
-
$\begingroup$ @JanKukacka.Thanks for the suggestion. I will check it out and come back. $\endgroup$– aneesh coolCommented Oct 4, 2018 at 8:32
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
The feature importance in Random Forest is based on out-of-bag error, and its value is>=0.
As per the Wikipedia page,
"To measure the importance of the j-th feature after training, the values of the j-th feature are permuted among the training data and the out-of-bag error is again computed on this perturbed data set. The importance score for the j-th feature is computed by averaging the difference in out-of-bag error before and after the permutation over all trees. The score is normalized by the standard deviation of these differences."
So essentially the importance estimates are normalized to add up to 100 for convenience.
-
$\begingroup$ Thanks a lot. So first we build the forest. Then we pass down the out of bag samples and permute each variables. Then we take difference between baseline accuracy and accuracy after the variables perturbed. Then we average the differences for a variable for all the trees and divide by its standard deviation. Am I right? $\endgroup$ Commented Oct 4, 2018 at 11:07
-
$\begingroup$ @AneshMuthiah Yes. You re right. You can find more info here eeecon.uibk.ac.at/~zeileis/papers/Lifestat-2008.pdf $\endgroup$ Commented Oct 5, 2018 at 7:50