How to actually plot a sample tree from randomForest::getTree()? [closed]

Question

Anyone got library or code suggestions on how to actually plot a couple of sample trees from:

getTree(rfobj, k, labelVar=TRUE)

(Yes I know you're not supposed to do this operationally, RF is a blackbox, etc etc. I want to visually sanity-check a tree to see if any variables are behaving counterintuitively, need tweaking/combining/discretization/transformation, check how well my encoded factors are working, etc.)

---

Prior questions without a decent answer:

• How to make Random Forests more interpretable?

• Also Obtaining knowledge from a random forest

I actually want to plot a sample tree. So don't argue with me about that, already. I'm not asking about varImpPlot(Variable Importance Plot) or partialPlot or MDSPlot, or these other plots, I already have those, but they're not a substitute for seeing a sample tree. Yes I can visually inspect the output of getTree(...,labelVar=TRUE).

(I guess a plot.rf.tree() contribution would be very-well-received.)

Answer 1

First (and easiest) solution: If you are not keen to stick with classical RF, as implemented in Andy Liaw's randomForest, you can try the party package which provides a different implementation of the original RF algorithm (use of conditional trees and aggregation scheme based on units weight average). Then, as reported on this R-help post, you can plot a single member of the list of trees. It seems to run smoothly, as far as I can tell. Below is a plot of one tree generated by cforest(Species ~ ., data=iris, controls=cforest_control(mtry=2, mincriterion=0)).

Second (almost as easy) solution: Most of tree-based techniques in R (tree, rpart, TWIX, etc.) offers a tree-like structure for printing/plotting a single tree. The idea would be to convert the output of randomForest::getTree to such an R object, even if it is nonsensical from a statistical point of view. Basically, it is easy to access the tree structure from a tree object, as shown below. Please note that it will slightly differ depending of the type of task--regression vs. classification--where in the later case it will add class-specific probabilities as the last column of the obj$frame (which is a data.frame).

> library(tree)
> tr <- tree(Species ~ ., data=iris)
> tr
node), split, n, deviance, yval, (yprob)
      * denotes terminal node

 1) root 150 329.600 setosa ( 0.33333 0.33333 0.33333 )  
   2) Petal.Length < 2.45 50   0.000 setosa ( 1.00000 0.00000 0.00000 ) *
   3) Petal.Length > 2.45 100 138.600 versicolor ( 0.00000 0.50000 0.50000 )  
     6) Petal.Width < 1.75 54  33.320 versicolor ( 0.00000 0.90741 0.09259 )  
      12) Petal.Length < 4.95 48   9.721 versicolor ( 0.00000 0.97917 0.02083 )  
        24) Sepal.Length < 5.15 5   5.004 versicolor ( 0.00000 0.80000 0.20000 ) *
        25) Sepal.Length > 5.15 43   0.000 versicolor ( 0.00000 1.00000 0.00000 ) *
      13) Petal.Length > 4.95 6   7.638 virginica ( 0.00000 0.33333 0.66667 ) *
     7) Petal.Width > 1.75 46   9.635 virginica ( 0.00000 0.02174 0.97826 )  
      14) Petal.Length < 4.95 6   5.407 virginica ( 0.00000 0.16667 0.83333 ) *
      15) Petal.Length > 4.95 40   0.000 virginica ( 0.00000 0.00000 1.00000 ) *
> tr$frame
            var   n        dev       yval splits.cutleft splits.cutright yprob.setosa yprob.versicolor yprob.virginica
1  Petal.Length 150 329.583687     setosa          <2.45           >2.45   0.33333333       0.33333333      0.33333333
2        <leaf>  50   0.000000     setosa                                  1.00000000       0.00000000      0.00000000
3   Petal.Width 100 138.629436 versicolor          <1.75           >1.75   0.00000000       0.50000000      0.50000000
6  Petal.Length  54  33.317509 versicolor          <4.95           >4.95   0.00000000       0.90740741      0.09259259
12 Sepal.Length  48   9.721422 versicolor          <5.15           >5.15   0.00000000       0.97916667      0.02083333
24       <leaf>   5   5.004024 versicolor                                  0.00000000       0.80000000      0.20000000
25       <leaf>  43   0.000000 versicolor                                  0.00000000       1.00000000      0.00000000
13       <leaf>   6   7.638170  virginica                                  0.00000000       0.33333333      0.66666667
7  Petal.Length  46   9.635384  virginica          <4.95           >4.95   0.00000000       0.02173913      0.97826087
14       <leaf>   6   5.406735  virginica                                  0.00000000       0.16666667      0.83333333
15       <leaf>  40   0.000000  virginica                                  0.00000000       0.00000000      1.00000000

Then, there are methods for pretty printing and plotting those objects. The key functions are a generic tree:::plot.tree method (I put a triple : which allows you to view the code in R directly) relying on tree:::treepl (graphical display) and tree:::treeco (compute nodes coordinates). These functions expect the obj$frame representation of the tree. Other subtle issues: (1) the argument type = c("proportional", "uniform") in the default plotting method, tree:::plot.tree, help to manage vertical distance between nodes (proportional means it is proportional to deviance, uniform mean it is fixed); (2) you need to complement plot(tr) by a call to text(tr) to add text labels to nodes and splits, which in this case means that you will also have to take a look at tree:::text.tree.

The getTree method from randomForest returns a different structure, which is documented in the online help. A typical output is shown below, with terminal nodes indicated by status code (-1). (Again, output will differ depending on the type of task, but only on the status and prediction columns.)

> library(randomForest)
> rf <- randomForest(Species ~ ., data=iris)
> getTree(rf, 1, labelVar=TRUE)
   left daughter right daughter    split var split point status prediction
1              2              3 Petal.Length        4.75      1       <NA>
2              4              5 Sepal.Length        5.45      1       <NA>
3              6              7  Sepal.Width        3.15      1       <NA>
4              8              9  Petal.Width        0.80      1       <NA>
5             10             11  Sepal.Width        3.60      1       <NA>
6              0              0         <NA>        0.00     -1  virginica
7             12             13  Petal.Width        1.90      1       <NA>
8              0              0         <NA>        0.00     -1     setosa
9             14             15  Petal.Width        1.55      1       <NA>
10             0              0         <NA>        0.00     -1 versicolor
11             0              0         <NA>        0.00     -1     setosa
12            16             17 Petal.Length        5.40      1       <NA>
13             0              0         <NA>        0.00     -1  virginica
14             0              0         <NA>        0.00     -1 versicolor
15             0              0         <NA>        0.00     -1  virginica
16             0              0         <NA>        0.00     -1 versicolor
17             0              0         <NA>        0.00     -1  virginica

If you can manage to convert the above table to the one generated by tree, you will probably be able to customize tree:::treepl, tree:::treeco and tree:::text.tree to suit your needs, though I do not have an example of this approach. In particular, you probably want to get rid of the use of deviance, class probabilities, etc. which are not meaningful in RF. All you want is to set up nodes coordinates and split values. You could use fixInNamespace() for that, but, to be honest, I'm not sure this is the right way to go.

Third (and certainly clever) solution: Write a true as.tree helper function which will alleviates all of the above "patches". You could then use R's plotting methods or, probably better, Klimt (directly from R) to display individual trees.

Answer 2

I'm four years late, but if you really want to stick to the randomForest package (and there are some good reasons to do so), and want to actually visualize the tree, you can use the reprtree package.

The package is not super well documented (you can find the docs here), but everything is pretty straightforward. To install the package refer to initialize.R in the repo, so simply run the following:

options(repos='http://cran.rstudio.org')
have.packages <- installed.packages()
cran.packages <- c('devtools','plotrix','randomForest','tree')
to.install <- setdiff(cran.packages, have.packages[,1])
if(length(to.install)>0) install.packages(to.install)

library(devtools)
if(!('reprtree' %in% installed.packages())){
   install_github('munoztd0/reprtree')
}
for(p in c(cran.packages, 'reprtree')) eval(substitute(library(pkg), list(pkg=p)))

Then go ahead and make your model and tree:

library(randomForest)
library(reprtree)

model <- randomForest(Species ~ ., data=iris, importance=TRUE, ntree=500, mtry = 2, do.trace=100)

reprtree:::plot.getTree(model)

And there you go! Beautiful and simple.

You can check the github repo to learn about the other methods in the package. In fact, if you check plot.getTree.R, you'll notice that the author uses his own implementation of as.tree() which chl♦ suggested you could build yourself in his answer. This means that you could do this:

tree <- getTree(model, k=1, labelVar=TRUE)
realtree <- reprtree:::as.tree(tree, model)

And then potentially use realtree with other tree plotting packages such as tree.

Answer 3

I've created some functions to extract the rules of a tree.

#***********************
#return the rules of a tree
#***********************
getConds<-function(tree){
  #store all conditions into a list
  conds<-list()
  #start by the terminal nodes and find previous conditions
  id.leafs<-which(tree$status==-1)
	  j<-0
	  for(i in id.leafs){
		j<-j+1
		prevConds<-prevCond(tree,i)
		conds[[j]]<-prevConds$cond
		while(prevConds$id>1){
		  prevConds<-prevCond(tree,prevConds$id)
		  conds[[j]]<-paste(conds[[j]]," & ",prevConds$cond)
        }
		if(prevConds$id==1){
			conds[[j]]<-paste(conds[[j]]," => ",tree$prediction[i])
    }
    
  }
  
  return(conds)
}

#**************************
#find the previous conditions in the tree
#**************************
prevCond<-function(tree,i){
  if(i %in% tree$right_daughter){
		id<-which(tree$right_daughter==i)
		cond<-paste(tree$split_var[id],">",tree$split_point[id])
	  }
	  if(i %in% tree$left_daughter){
    id<-which(tree$left_daughter==i)
		cond<-paste(tree$split_var[id],"<",tree$split_point[id])
  }
  
  return(list(cond=cond,id=id))
}

#remove spaces in a word
collapse<-function(x){
  x<-sub(" ","_",x)
  
  return(x)
}


data(iris)
require(randomForest)
mod.rf <- randomForest(Species ~ ., data=iris)
tree<-getTree(mod.rf, k=1, labelVar=TRUE)
#rename the name of the column
colnames(tree)<-sapply(colnames(tree),collapse)
rules<-getConds(tree)
print(rules)