# How to make Random Forests more interpretable? [duplicate]

Are there any methods that one could utilize to make Random Forest more interpretable? Random Forest performs much better than CART but it is a lot less interpretable.

• Did you take a look at the random-forest tag? In particular, this question and the accepted answer may be of interest to you: Obtaining knowledge from a random forest.
– chl
Jul 11, 2012 at 18:49
• A recent article in JCGS suggests a novel approach for visualizing relationships between predictors in random forests named Partition Maps (Meinshausen, 2011, pre-print PDF here. He also has an R package for the graphs on his website. Jul 11, 2012 at 20:18
• @AndyW (+1) Thanks for sharing those references.
– chl
Jul 11, 2012 at 20:22
• This is just... wrong. The core idea behind ML is that models are black boxes, thus looking inside will always be either deceiving or disappointing.
– user88
Jul 11, 2012 at 20:46
• @mbq My knowledge is quite limited, but this is ... a very strong statement. If one finds certain features to be useful for predicting a certain outcome both on the validation set AND in live application, these features contain useful information. Period. It may be that the data is flawed and it performed well just by accident or only in this specific case, but it gives you hints. Compare this hints across several projects in the same domain and one get's some sort of domain knowledge. Jul 16, 2012 at 11:53

The results from CART can change easily (with realistic sample sizes) with small perturbations to the data. If this is the case, it seems the interpretation is not a straightforward as it seems. I've often heard some of my colleagues avoiding random forests because of difficulties in interpretation. They are built more for prediction. Even the variable importance measures that come out are based on predictive performance, but they do help with interpretation.

• The first sentence is really on the point. Frank Harrell mentioned in a comment on this site something like $n>10,000$ to get stable results, if I remember correctly. The variable importance measures are often used for feature selection in wrapper or embedded methods, though. Maybe you want to elaborate on where difficulties with interpretation arise?
– chl
Jul 11, 2012 at 20:20

For each tree in the forest you have an interpretation for the terminal nodes. So the forest can be viewed as a series of explanations why vector x might belong to class y. Then the class with the largest number of reasonable explanations is the class that is picked. Isn't that fairly easy to understand?

• Sounds to me like you're explaining how the technique works overall, but the OP is wanting to take a trained Random Forest and summarize to someone what it is saying. Jul 11, 2012 at 19:41
• The Random forest algorithm is an ensemble method which uses random sampling of cases (bootstrap) and variables (mtry parameter). The out-of-bag sample is used to derive a measure of variable importance (using a permutation technique) and assess cases proximities through a voting process (number of times two individual ended up in the same leave divided by the number of trees). Viewed as a black box, I don't think RF yields such an easy interpretation (yet it outperforms many competive algorithms which are easier to decipher).
– chl
Jul 11, 2012 at 20:01
• Well in specific case you have specific rules for each tree and hence specific reasons for votes. That can be explained. But certainly when going through an ensemble of trees there is much greater complication than looking at just one. Jul 11, 2012 at 22:04
• If I were to give the RF to someone who knows nothing about machine learning, it would be difficult to identify unique rules as to why an object is classified in a particular way. Each tree may have contradictory rules. Is there anyway to train a single CART tree from the Random Forest? Jul 12, 2012 at 5:56
• Of course the forest has conflicting decisions. That is why the decision is made by majority rule. So-called training of a classifier in the context of trees is done by growing the tree and creating the forest. A constructed tree is equivalent to a fit LDF or QDF. Jul 12, 2012 at 10:24