73
$\begingroup$

Anyone got library or code suggestions on how to actually plot a couple of sample trees from:

getTree(rfobj, k, labelVar=TRUE)

(Yes I know you're not supposed to do this operationally, RF is a blackbox, etc etc. I want to visually sanity-check a tree to see if any variables are behaving counterintuitively, need tweaking/combining/discretization/transformation, check how well my encoded factors are working, etc.)


Prior questions without a decent answer:

I actually want to plot a sample tree. So don't argue with me about that, already. I'm not asking about varImpPlot(Variable Importance Plot) or partialPlot or MDSPlot, or these other plots, I already have those, but they're not a substitute for seeing a sample tree. Yes I can visually inspect the output of getTree(...,labelVar=TRUE).

(I guess a plot.rf.tree() contribution would be very-well-received.)

$\endgroup$
6
  • 8
    $\begingroup$ I don't see a need to be preemptively argumentative, especially if you are asking for someone to volunteer to help you; it doesn't come across well. CV has an etiquette policy--you may want to read our FAQ. $\endgroup$ Oct 29, 2012 at 20:55
  • 14
    $\begingroup$ @gung: every previous question on this topic has decayed into people insisting that it was not necessary, and indeed heretical, to plot a sample tree. Read the citations I gave. I'm looking for a sketch here of how to code plotting an rf tree. $\endgroup$
    – smci
    Oct 29, 2012 at 21:07
  • 3
    $\begingroup$ I see some answers where users are trying to be helpful & address the question, along w/ some comments questioning the premise of the idea (which, I honestly believe, are intended in a helpful spirit as well). It's certainly possible to acknowledge that some people will disagree w/o being testy. $\endgroup$ Oct 29, 2012 at 21:18
  • 4
    $\begingroup$ I see zero answers where anyone has ever plotted a tree, in over a year. I'm looking for a specific answer to that specific question. $\endgroup$
    – smci
    Oct 29, 2012 at 21:21
  • 1
    $\begingroup$ It is possible to plot a single tree built with cforest (in the party package). Otherwise, you'll have to convert the data.frame returned by randomForest::getTree to a tree-like object. $\endgroup$
    – chl
    Oct 30, 2012 at 10:43

3 Answers 3

52
$\begingroup$

First (and easiest) solution: If you are not keen to stick with classical RF, as implemented in Andy Liaw's randomForest, you can try the party package which provides a different implementation of the original RF algorithm (use of conditional trees and aggregation scheme based on units weight average). Then, as reported on this R-help post, you can plot a single member of the list of trees. It seems to run smoothly, as far as I can tell. Below is a plot of one tree generated by cforest(Species ~ ., data=iris, controls=cforest_control(mtry=2, mincriterion=0)).

enter image description here

Second (almost as easy) solution: Most of tree-based techniques in R (tree, rpart, TWIX, etc.) offers a tree-like structure for printing/plotting a single tree. The idea would be to convert the output of randomForest::getTree to such an R object, even if it is nonsensical from a statistical point of view. Basically, it is easy to access the tree structure from a tree object, as shown below. Please note that it will slightly differ depending of the type of task--regression vs. classification--where in the later case it will add class-specific probabilities as the last column of the obj$frame (which is a data.frame).

> library(tree)
> tr <- tree(Species ~ ., data=iris)
> tr
node), split, n, deviance, yval, (yprob)
      * denotes terminal node

 1) root 150 329.600 setosa ( 0.33333 0.33333 0.33333 )  
   2) Petal.Length < 2.45 50   0.000 setosa ( 1.00000 0.00000 0.00000 ) *
   3) Petal.Length > 2.45 100 138.600 versicolor ( 0.00000 0.50000 0.50000 )  
     6) Petal.Width < 1.75 54  33.320 versicolor ( 0.00000 0.90741 0.09259 )  
      12) Petal.Length < 4.95 48   9.721 versicolor ( 0.00000 0.97917 0.02083 )  
        24) Sepal.Length < 5.15 5   5.004 versicolor ( 0.00000 0.80000 0.20000 ) *
        25) Sepal.Length > 5.15 43   0.000 versicolor ( 0.00000 1.00000 0.00000 ) *
      13) Petal.Length > 4.95 6   7.638 virginica ( 0.00000 0.33333 0.66667 ) *
     7) Petal.Width > 1.75 46   9.635 virginica ( 0.00000 0.02174 0.97826 )  
      14) Petal.Length < 4.95 6   5.407 virginica ( 0.00000 0.16667 0.83333 ) *
      15) Petal.Length > 4.95 40   0.000 virginica ( 0.00000 0.00000 1.00000 ) *
> tr$frame
            var   n        dev       yval splits.cutleft splits.cutright yprob.setosa yprob.versicolor yprob.virginica
1  Petal.Length 150 329.583687     setosa          <2.45           >2.45   0.33333333       0.33333333      0.33333333
2        <leaf>  50   0.000000     setosa                                  1.00000000       0.00000000      0.00000000
3   Petal.Width 100 138.629436 versicolor          <1.75           >1.75   0.00000000       0.50000000      0.50000000
6  Petal.Length  54  33.317509 versicolor          <4.95           >4.95   0.00000000       0.90740741      0.09259259
12 Sepal.Length  48   9.721422 versicolor          <5.15           >5.15   0.00000000       0.97916667      0.02083333
24       <leaf>   5   5.004024 versicolor                                  0.00000000       0.80000000      0.20000000
25       <leaf>  43   0.000000 versicolor                                  0.00000000       1.00000000      0.00000000
13       <leaf>   6   7.638170  virginica                                  0.00000000       0.33333333      0.66666667
7  Petal.Length  46   9.635384  virginica          <4.95           >4.95   0.00000000       0.02173913      0.97826087
14       <leaf>   6   5.406735  virginica                                  0.00000000       0.16666667      0.83333333
15       <leaf>  40   0.000000  virginica                                  0.00000000       0.00000000      1.00000000

Then, there are methods for pretty printing and plotting those objects. The key functions are a generic tree:::plot.tree method (I put a triple : which allows you to view the code in R directly) relying on tree:::treepl (graphical display) and tree:::treeco (compute nodes coordinates). These functions expect the obj$frame representation of the tree. Other subtle issues: (1) the argument type = c("proportional", "uniform") in the default plotting method, tree:::plot.tree, help to manage vertical distance between nodes (proportional means it is proportional to deviance, uniform mean it is fixed); (2) you need to complement plot(tr) by a call to text(tr) to add text labels to nodes and splits, which in this case means that you will also have to take a look at tree:::text.tree.

The getTree method from randomForest returns a different structure, which is documented in the online help. A typical output is shown below, with terminal nodes indicated by status code (-1). (Again, output will differ depending on the type of task, but only on the status and prediction columns.)

> library(randomForest)
> rf <- randomForest(Species ~ ., data=iris)
> getTree(rf, 1, labelVar=TRUE)
   left daughter right daughter    split var split point status prediction
1              2              3 Petal.Length        4.75      1       <NA>
2              4              5 Sepal.Length        5.45      1       <NA>
3              6              7  Sepal.Width        3.15      1       <NA>
4              8              9  Petal.Width        0.80      1       <NA>
5             10             11  Sepal.Width        3.60      1       <NA>
6              0              0         <NA>        0.00     -1  virginica
7             12             13  Petal.Width        1.90      1       <NA>
8              0              0         <NA>        0.00     -1     setosa
9             14             15  Petal.Width        1.55      1       <NA>
10             0              0         <NA>        0.00     -1 versicolor
11             0              0         <NA>        0.00     -1     setosa
12            16             17 Petal.Length        5.40      1       <NA>
13             0              0         <NA>        0.00     -1  virginica
14             0              0         <NA>        0.00     -1 versicolor
15             0              0         <NA>        0.00     -1  virginica
16             0              0         <NA>        0.00     -1 versicolor
17             0              0         <NA>        0.00     -1  virginica

If you can manage to convert the above table to the one generated by tree, you will probably be able to customize tree:::treepl, tree:::treeco and tree:::text.tree to suit your needs, though I do not have an example of this approach. In particular, you probably want to get rid of the use of deviance, class probabilities, etc. which are not meaningful in RF. All you want is to set up nodes coordinates and split values. You could use fixInNamespace() for that, but, to be honest, I'm not sure this is the right way to go.

Third (and certainly clever) solution: Write a true as.tree helper function which will alleviates all of the above "patches". You could then use R's plotting methods or, probably better, Klimt (directly from R) to display individual trees.

$\endgroup$
0
52
$\begingroup$

I'm four years late, but if you really want to stick to the randomForest package (and there are some good reasons to do so), and want to actually visualize the tree, you can use the reprtree package.

The package is not super well documented (you can find the docs here), but everything is pretty straightforward. To install the package refer to initialize.R in the repo, so simply run the following:

options(repos='http://cran.rstudio.org')
have.packages <- installed.packages()
cran.packages <- c('devtools','plotrix','randomForest','tree')
to.install <- setdiff(cran.packages, have.packages[,1])
if(length(to.install)>0) install.packages(to.install)

library(devtools)
if(!('reprtree' %in% installed.packages())){
   install_github('munoztd0/reprtree')
}
for(p in c(cran.packages, 'reprtree')) eval(substitute(library(pkg), list(pkg=p)))

Then go ahead and make your model and tree:

library(randomForest)
library(reprtree)

model <- randomForest(Species ~ ., data=iris, importance=TRUE, ntree=500, mtry = 2, do.trace=100)

reprtree:::plot.getTree(model)

And there you go! Beautiful and simple.

tree generated from plot.getTree(model)

You can check the github repo to learn about the other methods in the package. In fact, if you check plot.getTree.R, you'll notice that the author uses his own implementation of as.tree() which chl♦ suggested you could build yourself in his answer. This means that you could do this:

tree <- getTree(model, k=1, labelVar=TRUE)
realtree <- reprtree:::as.tree(tree, model)

And then potentially use realtree with other tree plotting packages such as tree.

$\endgroup$
12
  • 8
    $\begingroup$ no problem. It took me hours to find the library/package so I figured that if not useful to you, it would be to other people trying to draw trees while still sticking to the randomForest package. $\endgroup$
    – jgozal
    Oct 22, 2016 at 5:49
  • 4
    $\begingroup$ Cool finding. Note: It plots the representative tree, in some sense, the tree in the ensemble which are on average the "closest" to all the other trees in the ensemble $\endgroup$
    – Chris
    Jun 2, 2017 at 20:21
  • 3
    $\begingroup$ @Chris The function plot.getTree() plots an individual tree. The function plot.reprtree() in that package plots a representative tree. $\endgroup$
    – Chun Li
    Mar 4, 2018 at 1:53
  • 1
    $\begingroup$ i got model from caret and want to feed into reptree with reprtree:::plot.getTree(mod_rf_1$finalModel), however, there's an "Error in data.frame(var = fr$var, splits = as.character(gTree[, "split point"]), : arguments imply differing number of rows: 2631, 0" $\endgroup$ Apr 19, 2018 at 2:00
  • 1
    $\begingroup$ Any issue that is directly related to the library/package should be directed to the author. Please open an issue on the github repo: github.com/araastat/reprtree/issues $\endgroup$
    – jgozal
    Dec 4, 2018 at 19:28
16
$\begingroup$

I've created some functions to extract the rules of a tree.

#***********************
#return the rules of a tree
#***********************
getConds<-function(tree){
  #store all conditions into a list
  conds<-list()
  #start by the terminal nodes and find previous conditions
  id.leafs<-which(tree$status==-1)
	  j<-0
	  for(i in id.leafs){
		j<-j+1
		prevConds<-prevCond(tree,i)
		conds[[j]]<-prevConds$cond
		while(prevConds$id>1){
		  prevConds<-prevCond(tree,prevConds$id)
		  conds[[j]]<-paste(conds[[j]]," & ",prevConds$cond)
        }
		if(prevConds$id==1){
			conds[[j]]<-paste(conds[[j]]," => ",tree$prediction[i])
    }
    
  }
  
  return(conds)
}

#**************************
#find the previous conditions in the tree
#**************************
prevCond<-function(tree,i){
  if(i %in% tree$right_daughter){
		id<-which(tree$right_daughter==i)
		cond<-paste(tree$split_var[id],">",tree$split_point[id])
	  }
	  if(i %in% tree$left_daughter){
    id<-which(tree$left_daughter==i)
		cond<-paste(tree$split_var[id],"<",tree$split_point[id])
  }
  
  return(list(cond=cond,id=id))
}

#remove spaces in a word
collapse<-function(x){
  x<-sub(" ","_",x)
  
  return(x)
}


data(iris)
require(randomForest)
mod.rf <- randomForest(Species ~ ., data=iris)
tree<-getTree(mod.rf, k=1, labelVar=TRUE)
#rename the name of the column
colnames(tree)<-sapply(colnames(tree),collapse)
rules<-getConds(tree)
print(rules)
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.