Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I am using the random forest algorithm as a robust classifier of two groups in a microarray study with 1000s of features.

  • What is the best way to present the random forest so that there is enough information to make it reproducible in a paper?
  • Is there a plot method in R to actually plot the tree, if there are a small number of features?
  • Is the OOB estimate of error rate the best statistic to quote?

Many thanks

share|improve this question
2  
There is not a single tree... But see @Shane's response for plotting one of them for illustrative purpose. –  chl Sep 3 '10 at 14:48
add comment

3 Answers 3

up vote 26 down vote accepted

Regarding making it reproducible, the best way is to provide reproducible research (i.e. code and data) along with the paper. Make it available on your website, or on a hosting site (like github).

Regarding visualization, Leo Breiman has done some interesting work on this (see his homepage, in particular the section on graphics).

But if you're using R, then the randomForest package has some useful functions:

data(mtcars)
mtcars.rf <- randomForest(mpg ~ ., data=mtcars, ntree=1000, keep.forest=FALSE,
                           importance=TRUE)
plot(mtcars.rf, log="y")
varImpPlot(mtcars.rf)

And

set.seed(1)
data(iris)
iris.rf <- randomForest(Species ~ ., iris, proximity=TRUE,
                        keep.forest=FALSE)
MDSplot(iris.rf, iris$Species)

I'm not aware of a simple way to actually plot a tree, but you can use the getTree function to retrieve the tree and plot that separately.

getTree(randomForest(iris[,-5], iris[,5], ntree=10), 3, labelVar=TRUE)

The Strobl/Zeileis presentation on "Why and how to use random forest variable importance measures (and how you shouldn’t)" has examples of trees which must have been produced in this way. This blog post on tree models has some nice examples of CART tree plots which you can use for example.

As @chl commented, a single tree isn't especially meaningful in this context, so short of using it to explain what a random forest is, I wouldn't include this in a paper.

share|improve this answer
2  
Small extension about plots: plot.randomForest shows how OOB error and in-class OOB error evolved with increasing number of trees; varImpPlot shows attribute importance measures for top attributes and MDSplot all objects plotted on the 2D projection of RF object proximity measure. –  mbq Sep 3 '10 at 19:25
    
+1 for citing the MDSplot() function. I must admit I often use RFs as a way to highlight clusters of individuals (based on the RF proximity measure) rather than selecting the best features. Clinicians often read much easily such plots than dotplot of var. importance... –  chl Sep 3 '10 at 20:07
add comment
  1. As Shane wrote; make it reproducible research + include random seeds, because RF is stochastic.
  2. First of all, plotting single trees forming RF is nonsense; this is an ensemble classifier, it makes sense only as a whole. But even plotting the whole forest is nonsense -- it is a black-box classifier, so it is not intended to explain the data with its structure, rather to replicate the original process. Instead, make some of plots Shane suggested.
  3. In practice, OOB is a very good error approximation; yet this is not a widely accepted fact, so for publication it is better to also make a CV to confirm it.
share|improve this answer
1  
+1 Great point about setting the seed values. –  Shane Sep 3 '10 at 19:32
1  
@danielsbrewer I would do this in some other way (paying more attention to feature selection), but this is correct; yet it is more on the topic of benchmarking RF feature selection than on selecting best markers for your biological problem. –  mbq Sep 6 '10 at 17:39
2  
The main problem is that it is really hard to compare two models (model=learning method+feature selection method), but for simplicity you can just assume something (like I'll use RF and select top 10 attributes) and admit that you know that this may be suboptimal, but you agree on that while you are for instance satisfied with the accuracy. In that case your only problem is to remove bias of attribute selection. tbc. –  mbq Sep 7 '10 at 23:53
2  
So, I would do a simple bagging: you create 10 (or 30 if you have a good computer) random subsamples of objects (let's say by random picking with replacement), train RF on each, get it's importance and return a rank of each attribute averaged over all repetitions (best attribute gets rank 1, second best 2 and so on; it can be averaged so the attribute that was 12 times 1st and 18 times 2nd have rank of 1.6), finally select 10 with best ranks and call them your markers. Then use a CV (LOO, 10-fold or preferably random sampling) to obtain an error approximation of RF using your markers. tbc. –  mbq Sep 8 '10 at 0:00
2  
Report the ranks (hopefully they should be pretty near 1,2,3...), CV error with its deviation (just count standard deviation of results of each CV round) and OOB error (probably will be identical to CV error). DISCLAIMER: This is not a method for selecting optimal number of attributes -- you need RFE and nested CV to do that. DISCLAIMER2: I haven't worked with such a data, so I don't guarantee that your referees will be happy with it (though I believe they should). –  mbq Sep 8 '10 at 0:11
show 2 more comments

Keep in mind the caveats in the other answers about the plot necessarily being meaningful. But if you want a plot for illustrative/pedagogical purposes, the following snippet of R might be useful. Not hard to add "split point" to the edge text if you need it.

to.dendrogram <- function(dfrep,rownum=1,height.increment=0.1){

  if(dfrep[rownum,'status'] == -1){
    rval <- list()

    attr(rval,"members") <- 1
    attr(rval,"height") <- 0.0
    attr(rval,"label") <- dfrep[rownum,'prediction']
    attr(rval,"leaf") <- TRUE

  }else{##note the change "to.dendrogram" and not "to.dendogram"
    left <- to.dendrogram(dfrep,dfrep[rownum,'left daughter'],height.increment)
    right <- to.dendrogram(dfrep,dfrep[rownum,'right daughter'],height.increment)
    rval <- list(left,right)

    attr(rval,"members") <- attr(left,"members") + attr(right,"members")
    attr(rval,"height") <- max(attr(left,"height"),attr(right,"height")) + height.increment
    attr(rval,"leaf") <- FALSE
    attr(rval,"edgetext") <- dfrep[rownum,'split var']
  }

  class(rval) <- "dendrogram"

  return(rval)
}

mod <- randomForest(Species ~ .,data=iris)
tree <- getTree(mod,1,labelVar=TRUE)

d <- to.dendrogram(tree)
str(d)
plot(d,center=TRUE,leaflab='none',edgePar=list(t.cex=1,p.col=NA,p.lty=0))
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.