3
$\begingroup$

When using R randomForest package I use replace=TRUE, which then dictates to:

if (replace) nrow(x) else ceiling(.632*nrow(x))

I was wondering if anyone knows of published papers/arguments for the pro's and con's (specifically relating to RF variable importance) of using replace=TRUE versus replace=FALSE, where replace=FALSE results in the conventional OOB error.

Thanks!

$\endgroup$

2 Answers 2

3
$\begingroup$

RF evolved from bagging/bootstrapping-based methods, so sampling with replacement=true seems to be its rather integral feature. To quote Wiki: "it leads to better model performance because it decreases the variance of the model, without increasing the bias". You could use some of the original bootstrapping refs to support this eg http://en.wikipedia.org/wiki/Bootstrapping_(statistics) However, there are cons eg Strobl et al. demonstrating how it can be used to mitigate method bias due to variable number of categories across different predictors http://www.statistik.lmu.de/~carolin/research/varimppaper_techreport.pdf

$\endgroup$
1
  • 1
    $\begingroup$ Thanks this paper looks like a great ref. I look forward to reading it! $\endgroup$
    – SOUser
    Apr 10, 2015 at 17:18
3
$\begingroup$

1 - The randomForest package $inbag matrix do not keep count of which observations were sampled twice, only if they're inbag (1) or OOB (0). Therefore to reconstruct the exact bootstrap, replacement must be set to FALSE. cinbag (package trimTrees) is a copy of randomForest which fixes this by adding a count inbag matrix.

update 15th July 2016: randomForest package has been providing actual inbag counts for some time now. No need for cinbag anymore.

2 - For models where OOB explained variance is lower than ~50%, it is advantageous to lower sampsize to decorrelate trees even further to ensure robust models. For optimal models it then no longer matters much weather replacement is used or not.

enter image description here

.

  #data set
  vars = 6
  obs  = 2000
  noise.factor = 3
  X = data.frame(replicate(6,rnorm(obs)))
  ysignal = with(X,sin(X1)+X2^2+X3+abs(X4)^.5)
  Y = ysignal + rnorm(obs,sd=sd(ysignal)*noise.factor)
  print(cor(ysignal,Y))

  #grid search light
  replaces= c(replace=F,replace=T)
  sampsizes = c(50,70,90,110,130,150,170,200,250,300,400,600,1000,1500,1900)
  pars  =  expand.grid(replace = replaces,
                       sampsize=sampsizes)
  #those paremeters changing
  parsInList = lapply(1:dim(pars)[1], function(i) pars[i,])
  #those paremeters constant
  std.arg = alist(x=X,y=Y,ntree=2000)

  library(parallel) #mclapply
  library(randomForest)
  R2.fit = mclapply(parsInList, function(these.pars) {
    run.arg = c(std.arg,these.pars)      
    rfo=do.call(what = randomForest, args = run.arg)
    print(rfo$call)
    rev(rfo$rsq)[1]
  })

  #plot results
  R2.fit.m = matrix(unlist(R2.fit),nrow=length(replaces))
  plot(sampsizes,R2.fit.m[1,],type="l",col="green",ylim=range(R2.fit.m),
  ylab="OOB %variance explained, pseudo R2",main=c("red: replace=T green: replace=F"))
  points(sampsizes,R2.fit.m[2,],type="l",col="red")
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.