Let's say I'm using a random forest in a true/false classification problem.

When I produce a score for an observation is it possible to get some sort of metric of variable importance for that particular score?

I'd imagine this would be based on the observation's leaf locations across the ensemble.


Had you tried the importance function of the randomForest package? For example:

x <- randomForest(Species ~ ., data=iris, importance=TRUE,
importance(x, type = 2)
# MeanDecreaseGini
# Sepal.Length         9.223181
# Sepal.Width          2.418925
# Petal.Length        40.900372
# Petal.Width         46.665629
  • $\begingroup$ That gives the importance for the overall model. My problem is, once you start scoring individual instances, can you get some measure of feature importance by instance? I thought I read somewhere that this might be possible with an alternating decision tree, but can't find the reference anymore. $\endgroup$ – Luke Dec 16 '13 at 19:21
  • $\begingroup$ how so? Why couldn't you base importance on an observations path through the tree? $\endgroup$ – Luke Mar 7 '14 at 0:57
  • $\begingroup$ True, but what about an aggregation of the values derived from the paths followed by an observation? $\endgroup$ – Luke Mar 7 '14 at 1:20
  • $\begingroup$ @Luke somehow, you want to figure out which variables played the most important roles in generating the prediction for a given observation? $\endgroup$ – Antoine Aug 17 '15 at 9:38
  • $\begingroup$ Yeah. That's what I'm looking for. $\endgroup$ – Luke Aug 17 '15 at 17:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.