# Best way to present a random forest in a publication?

I am using the random forest algorithm as a robust classifier of two groups in a microarray study with 1000s of features.

• What is the best way to present the random forest so that there is enough information to make it reproducible in a paper?
• Is there a plot method in R to actually plot the tree, if there are a small number of features?
• Is the OOB estimate of error rate the best statistic to quote?
• There is not a single tree... But see @Shane's response for plotting one of them for illustrative purpose.
– chl
Sep 3, 2010 at 14:48
• I certainly worth considering randomForest::partialPlot, stats.stackexchange.com/questions/92150/… Feb 7, 2016 at 21:11
• you could try my random forest visualization package, forestFloor - forestfloor.dk Jun 1, 2016 at 11:56

Regarding making it reproducible, the best way is to provide reproducible research (i.e. code and data) along with the paper. Make it available on your website, or on a hosting site (like github).

Regarding visualization, Leo Breiman has done some interesting work on this (see his homepage, in particular the section on graphics).

But if you're using R, then the randomForest package has some useful functions:

data(mtcars)
mtcars.rf <- randomForest(mpg ~ ., data=mtcars, ntree=1000, keep.forest=FALSE,
importance=TRUE)
plot(mtcars.rf, log="y")
varImpPlot(mtcars.rf)


And

set.seed(1)
data(iris)
iris.rf <- randomForest(Species ~ ., iris, proximity=TRUE,
keep.forest=FALSE)
MDSplot(iris.rf, iris\$Species)


I'm not aware of a simple way to actually plot a tree, but you can use the getTree function to retrieve the tree and plot that separately.

getTree(randomForest(iris[,-5], iris[,5], ntree=10), 3, labelVar=TRUE)


The Strobl/Zeileis presentation on "Why and how to use random forest variable importance measures (and how you shouldn’t)" has examples of trees which must have been produced in this way. This blog post on tree models has some nice examples of CART tree plots which you can use for example.

As @chl commented, a single tree isn't especially meaningful in this context, so short of using it to explain what a random forest is, I wouldn't include this in a paper.

• Small extension about plots: plot.randomForest shows how OOB error and in-class OOB error evolved with increasing number of trees; varImpPlot shows attribute importance measures for top attributes and MDSplot all objects plotted on the 2D projection of RF object proximity measure.
– user88
Sep 3, 2010 at 19:25
• +1 for citing the MDSplot() function. I must admit I often use RFs as a way to highlight clusters of individuals (based on the RF proximity measure) rather than selecting the best features. Clinicians often read much easily such plots than dotplot of var. importance...
– chl
Sep 3, 2010 at 20:07
1. As Shane wrote; make it reproducible research + include random seeds, because RF is stochastic.
2. First of all, plotting single trees forming RF is nonsense; this is an ensemble classifier, it makes sense only as a whole. But even plotting the whole forest is nonsense -- it is a black-box classifier, so it is not intended to explain the data with its structure, rather to replicate the original process. Instead, make some of plots Shane suggested.
3. In practice, OOB is a very good error approximation; yet this is not a widely accepted fact, so for publication it is better to also make a CV to confirm it.
• So @mbq when doing a CV is it valid to firstly do a random forest with all samples selected; doing it twice once with all and secondly with the top 10 variables (which can be quoted in a paper). Then do a leave-one out cross-validation (selecting the 10 top genes each try) and quote the CV error from that? Sep 6, 2010 at 15:47
• @danielsbrewer I would do this in some other way (paying more attention to feature selection), but this is correct; yet it is more on the topic of benchmarking RF feature selection than on selecting best markers for your biological problem.
– user88
Sep 6, 2010 at 17:39
• The main problem is that it is really hard to compare two models (model=learning method+feature selection method), but for simplicity you can just assume something (like I'll use RF and select top 10 attributes) and admit that you know that this may be suboptimal, but you agree on that while you are for instance satisfied with the accuracy. In that case your only problem is to remove bias of attribute selection. tbc.
– user88
Sep 7, 2010 at 23:53
• So, I would do a simple bagging: you create 10 (or 30 if you have a good computer) random subsamples of objects (let's say by random picking with replacement), train RF on each, get it's importance and return a rank of each attribute averaged over all repetitions (best attribute gets rank 1, second best 2 and so on; it can be averaged so the attribute that was 12 times 1st and 18 times 2nd have rank of 1.6), finally select 10 with best ranks and call them your markers. Then use a CV (LOO, 10-fold or preferably random sampling) to obtain an error approximation of RF using your markers. tbc.
– user88
Sep 8, 2010 at 0:00
• Report the ranks (hopefully they should be pretty near 1,2,3...), CV error with its deviation (just count standard deviation of results of each CV round) and OOB error (probably will be identical to CV error). DISCLAIMER: This is not a method for selecting optimal number of attributes -- you need RFE and nested CV to do that. DISCLAIMER2: I haven't worked with such a data, so I don't guarantee that your referees will be happy with it (though I believe they should).
– user88
Sep 8, 2010 at 0:11

Keep in mind the caveats in the other answers about the plot necessarily being meaningful. But if you want a plot for illustrative/pedagogical purposes, the following snippet of R might be useful. Not hard to add "split point" to the edge text if you need it.

to.dendrogram <- function(dfrep,rownum=1,height.increment=0.1){

if(dfrep[rownum,'status'] == -1){
rval <- list()

attr(rval,"members") <- 1
attr(rval,"height") <- 0.0
attr(rval,"label") <- dfrep[rownum,'prediction']
attr(rval,"leaf") <- TRUE

}else{##note the change "to.dendrogram" and not "to.dendogram"
left <- to.dendrogram(dfrep,dfrep[rownum,'left daughter'],height.increment)
right <- to.dendrogram(dfrep,dfrep[rownum,'right daughter'],height.increment)
rval <- list(left,right)

attr(rval,"members") <- attr(left,"members") + attr(right,"members")
attr(rval,"height") <- max(attr(left,"height"),attr(right,"height")) + height.increment
attr(rval,"leaf") <- FALSE
attr(rval,"edgetext") <- dfrep[rownum,'split var']
#To add Split Point in Dendrogram
#attr(rval,"edgetext") <- paste(dfrep[rownum,'split var'],"\n<",round(dfrep[rownum,'split point'], digits = 2),"=>", sep = " ")
}

class(rval) <- "dendrogram"

return(rval)
}

mod <- randomForest(Species ~ .,data=iris)
tree <- getTree(mod,1,labelVar=TRUE)

d <- to.dendrogram(tree)
str(d)
plot(d,center=TRUE,leaflab='none',edgePar=list(t.cex=1,p.col=NA,p.lty=0))

• The code produces very good tree plot. But the values are not being displayed. Probably a text() function needs to be added after last (plot) statement.
– rnso
Apr 12, 2015 at 2:59