Feature importance for random forest classification of a sample

Question

This is a cross post from : https://datascience.stackexchange.com/questions/8208/feature-importance-for-random-forest-classification-of-a-sample

Using a random forest is it possible to determine which features were the driving features to classify a specific sample as class A?

I know I can ask which features are more important to perform classification of ANY sample, but can I ask this for a specific sample? E.g. Why was sample 1 classified as A? Which of its features were much more like class A than class B?

Does it even make sense to ask this question of a random forest?

Bonus points on how to do it with sklearn in python :)

Answer 1

Variable importance accounts for the increase in out-of-bag cross-validated prediction error. It would be possible but not meaningful to account for the change of prediction error by one sample only. As one sample only can be correctly or wrongly predicted, such a term would be very unstable and crude.

You could check out 'local variable importance', 'partial dependence plots' or 'feature contributions'. Here's an example from my package forestFloor using feature contributions. Each plot shows the change of predicted class probability as function each variable. For the iris data set, there no strong variable interactions. Therefore, the model structure can be boiled down to a 2D visualization. The R-sqaured terms quantifies how much the model structure deviates from this main effect only interpretation/visualization.

library(forestFloor)
library(randomForest)
data(iris)
X = iris[,!names(iris) %in% "Species"]
Y = iris[,"Species"]

rf = randomForest(X,Y,
                  keep.forest=TRUE, #mandatory for classification
                  replace=FALSE,    #if TRUE use trimTrees::cinbag, not randomForest
                  keep.inbag=TRUE,  #mandatory always for forestFloor
                  sampsize =15 )    #optional:smaller trees smoother model structure

ff = forestFloor(rf.fit  = rf,           # mandatory
                 X       = X,            # mandatory
                 calc_np = "sad monkey", # this input takes no effect for classification
                 binary_reg = FALSE)     # can change two class classification to regression
# Thus cannot be TRUE for IRIS (three class)

plot(ff,plot_GOF=TRUE,cex=.7,
     colLists=list(c("#FF0000A5"),
                   c("#00FF0050"),
                   c("#0000FF35")))