Are there any methods that one could utilize to make Random Forest more interpretable? Random Forest performs much better than CART but it is a lot less interpretable.

  • 3
    Did you take a look at the random-forest tag? In particular, this question and the accepted answer may be of interest to you: Obtaining knowledge from a random forest. – chl Jul 11 '12 at 18:49
  • 3
    A recent article in JCGS suggests a novel approach for visualizing relationships between predictors in random forests named Partition Maps (Meinshausen, 2011, pre-print PDF here. He also has an R package for the graphs on his website. – Andy W Jul 11 '12 at 20:18
  • @AndyW (+1) Thanks for sharing those references. – chl Jul 11 '12 at 20:22
  • 1
    This is just... wrong. The core idea behind ML is that models are black boxes, thus looking inside will always be either deceiving or disappointing. – mbq Jul 11 '12 at 20:46
  • 1
    @mbq My knowledge is quite limited, but this is ... a very strong statement. If one finds certain features to be useful for predicting a certain outcome both on the validation set AND in live application, these features contain useful information. Period. It may be that the data is flawed and it performed well just by accident or only in this specific case, but it gives you hints. Compare this hints across several projects in the same domain and one get's some sort of domain knowledge. – steffen Jul 16 '12 at 11:53
up vote 7 down vote accepted

The results from CART can change easily (with realistic sample sizes) with small perturbations to the data. If this is the case, it seems the interpretation is not a straightforward as it seems. I've often heard some of my colleagues avoiding random forests because of difficulties in interpretation. They are built more for prediction. Even the variable importance measures that come out are based on predictive performance, but they do help with interpretation.

  • 2
    The first sentence is really on the point. Frank Harrell mentioned in a comment on this site something like $n>10,000$ to get stable results, if I remember correctly. The variable importance measures are often used for feature selection in wrapper or embedded methods, though. Maybe you want to elaborate on where difficulties with interpretation arise? – chl Jul 11 '12 at 20:20

For each tree in the forest you have an interpretation for the terminal nodes. So the forest can be viewed as a series of explanations why vector x might belong to class y. Then the class with the largest number of reasonable explanations is the class that is picked. Isn't that fairly easy to understand?

  • Sounds to me like you're explaining how the technique works overall, but the OP is wanting to take a trained Random Forest and summarize to someone what it is saying. – Wayne Jul 11 '12 at 19:41
  • The Random forest algorithm is an ensemble method which uses random sampling of cases (bootstrap) and variables (mtry parameter). The out-of-bag sample is used to derive a measure of variable importance (using a permutation technique) and assess cases proximities through a voting process (number of times two individual ended up in the same leave divided by the number of trees). Viewed as a black box, I don't think RF yields such an easy interpretation (yet it outperforms many competive algorithms which are easier to decipher). – chl Jul 11 '12 at 20:01
  • 1
    Well in specific case you have specific rules for each tree and hence specific reasons for votes. That can be explained. But certainly when going through an ensemble of trees there is much greater complication than looking at just one. – Michael Chernick Jul 11 '12 at 22:04
  • If I were to give the RF to someone who knows nothing about machine learning, it would be difficult to identify unique rules as to why an object is classified in a particular way. Each tree may have contradictory rules. Is there anyway to train a single CART tree from the Random Forest? – lord12 Jul 12 '12 at 5:56
  • 1
    Of course the forest has conflicting decisions. That is why the decision is made by majority rule. So-called training of a classifier in the context of trees is done by growing the tree and creating the forest. A constructed tree is equivalent to a fit LDF or QDF. – Michael Chernick Jul 12 '12 at 10:24

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.