Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 9 down vote favorite
4

Say I've got a predictive classification model based on a random forest (using the randomForest package in R). I'd like to set it up so that end-users can specify an item to generate a prediction for, and it'll output a classification likelihood. So far, no problem.

But it would be useful/cool to be able to output something like a variable importance graph, but for the specific item being predicted, not for the training set. Something like:

Item X is predicted to be a Dog (73% likely)
Because:
Legs=4
Breath=bad
Fur=short
Food=nasty

You get the point. Is there a standard, or at least justifiable, way of extracting this information from a trained random forest? If so, does anyone have code that will do this for the randomForest package?

share|improve this question
Some complexity... You could imagine counting the number of times the Legs variable was part of the decision path. But would you just do that for trees that predicted the majority answer, or all of them? Or the difference? – Harlan Apr 7 '11 at 23:53
And changing all of the m predictors one by one and looking to see how the forest predicts differently seems sorta expensive. There's gotta be a better way. – Harlan Apr 8 '11 at 1:03
my first thought was to wonder how what you are wanting to do differs from the variable importance of the training data? Are you looking to say that given the other values as they were, what was the sensitivity of the prediction on legs =4 versus legs=2 or legs=0? Have you looked at the partial plot function in the randomforest package? – B_Miner Apr 8 '11 at 1:22
The variable importance is usually defined conditional on the whole training set (or the assumed population, or something). But what I want is the variable importance for a single predicted item. Imagine a case where the forest consists of very lopsided decision trees. Test Instance 1 could be explained by 1 or very few decision nodes, while Test Instance 2 could be explained by a much larger set of decision nodes. I want a very simple human-interpretable version of that, like a ranked set of decisions, the top 5 of which I can provide. For a single decision tree, I'd just read them off. – Harlan Apr 8 '11 at 1:59

1 Answer

up vote 2 down vote

First idea is just to mimic the knock-out strategy from variable importance and just test how mixing each attribute will degenerate the forest confidence in object classification (on OOB and with some repetitions obviously). This requires some coding, but is certainly achievable.

However, I feel it is just a bad idea -- the result will be probably variable like hell (without stabilizing impact of averaging over objects), noisy (for not-so-confident objects the nonsense attributes could have big impacts) and hard to interpret (two or more attribute cooperative rules will probably result in random impacts of each contributing attributes).

Not to leave you with negative answer, I would rather try to look at the proximity matrix and the possible archetypes it may reveal -- this seems much more stable and straightforward.

share|improve this answer
The cooperative rules/correlated predictors point is an excellent criticism. To make this work, it might be necessary to train the RF on some sort of pre-reduced set of variables, or incorporate some sort of penalization strategy to cause the RF to focus on a subset of predictors. – Harlan Mar 9 at 15:02

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.