How can I represent a randomForest object in a way that "unlocks the black box" in R? [duplicate]

Question

I have a randomForest object in R, and am trying to extract prescriptive insights, as I would with a tree. This is for a binary classification problem.

Given this setting, my main question is: How can I make sense of what values of what features are associated with each classification? For example, is there a way I can analyze the randomForest object to understand that values of feature1 between 0.5 and 0.8 are likely associated with a classification of "1"?

I would imagine there is something I could do using the votes for each object (given its feature values), or the feature importance for each feature in the forest?

Answer 1

For a variety of reasons, this is very hard to do for random forests, or most blackbox methods with complex interactions. Remember that a random forest is basically taking the average over a large number of decisions trees that are all trained to perfectly fit their subset of the data. Understanding even a single one of these saturated trees is nearly impossible, and now to "understand" the average of a large number of them is not a reasonable human task.

With that in mind, the simplest approach I would advise is to use a model in which the feature effects are much easier to understand, like an elastic net model. If the predictive ability is close to that of the random forest, then one might pick the elastic net model just due to interpretability over a random forest, even though the random forest might have slightly higher accuracy.

Finally, if one does find that the interpretable models are unacceptably worse in predictions than the non-interpretable model, then one way to answer your question is to simply plug feature values into your fitted model and see how the outcome changes. However, this is actually slightly more challenging than it looks; remember that in a random forest, the model allows for interactions between variables. So it is possible that if $x_1 = 0$, high values of $x_2$ lead to high probabilities of success in the outcome, but if $x_1 = 1$, low values of $x_2$ lead to high probabilities of success in the outcome.

One way to summarize this might be to plug in all rows of your data, except that you change your feature of interest into something like "low", "medium" and "high". Then compare the estimated probabilities given the different levels for your feature given several representative values of all the other covariates.

Finally, just a word of caution in general when interpreting feature effects from a predictive model. Note that you are interpreting how the model makes a decision, which is different than the actual relation between a feature and the outcome. For example, if there's very little evidence for an effect from a given feature, then the elastic net model will likely set this estimated coefficient to 0 due to the $L_1$ penalty term. However, lack of evidence is not the same as evidence of lack; if $x_1 = 0$ for 99.9% of your data, then you have very little evidence about the effect of $x_1$, and so the elastic net model is likely to set this coefficient to 0, even though the effect may be large.

Answer 2

There are a number of model-agnostic ways of interpreting complex machine learning models. Model-agnostic means that it can be applied to all sorts of models, including the random forest.

Christoph Molnar put his entire book Interpretable Machine Learning: A Guide for Making Black Box Models Explainable on the internet, and he devotes an entire chapter to model-agnostic methods.

Two popular methods are LIME (which stands for local interpretable model-agnostic explanations) and Shapley values. Both of these are described in the Molnar book listed above.

Crudely put, LIME looks at a specific prediction, makes slight variations on your data to see predictions around that space and to learn about how the model makes predictions in that local space, and then trains an interpretable model (such as linear regression) based on this information. I'm not recalling precisely how Shapley values work—other than remembering it is a clever extrapolation of a metric from game theory: Features are treated as cooperating in making predictions, and features receive a higher Shapley value for contributing to the prediction more.

I would suggest reading the chapters in Molnar's book above. I would also suggest, for a more conceptual overview:

• A talk of Molnar's on these methods, found at YouTube.

• The Data Skeptic podcast episode on LIME.

• Two episodes of the Linear Digressions podcast on Shapley values. The two webpages (here and here) have lots of other links to the original papers, use-case write-ups, and other episodes of theirs on LIME and model interpretation.

Apologies for not going too in depth on these methods, but these resources should point you in the right direction.