After following the questions

Below there is a reproducible example using IRIS dataset where I can't reproduce the random forest model when not using caret's train function.

library(doParallel); library(caret)

#create a list of seed, here change the seed for each resampling
seeds <- vector(mode = "list", length = 11)#length is = (n_repeats*nresampling)+1
for(i in 1:10) seeds[[i]]<- sample.int(n=1000, 3) #(3 is the number of tuning parameter, mtry for rf, here equal to ncol(iris)-2)

seeds[[11]]<-sample.int(1000, 1)#for the last model

#control list
myControl <- trainControl(method='cv', seeds = seeds, index=createFolds(iris$Species))

#run model in parallel
cl <- makeCluster(12, type="SOCK",outfile="")
model1 <- train(Species~., iris, method='rf', trControl=myControl)

model2 <- train(Species~., iris, method='rf', trControl=myControl)

all.equal(predict(model1, type='prob'), predict(model2, type='prob'))


 >[1] TRUE


# using the same seed as for model 2
model3 <- randomForest(Species~., iris, mtry = model2$bestTune$mtry)
all.equal(predict(model2, type='prob'),as.data.frame(predict(model3, type='prob')))


>[1] "Component “versicolor”: Mean relative difference: 0.3006435"
>[2] "Component “virginica”: Mean relative difference: 0.2400822" 
  • What am I missing ? Is it possible to reproduce the random forest model returned by caret's train function?
  • If not, can we train the final model in caret with different parameters ? such as importance and proximities flags to true ?

closed as off-topic by COOLSerdash, Greenparker, whuber Jul 18 '16 at 17:57

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – COOLSerdash, Greenparker, whuber
If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ Removing the parallel (i.e. every line containing cl) makes the result reproducible $\endgroup$ – RUser4512 Dec 30 '15 at 15:55
  • $\begingroup$ Sorry, but it is no so. Making it sequential does not make it reproducible. $\endgroup$ – sc25 Dec 30 '15 at 19:56
  • $\begingroup$ Strange, it worked on my machine... I will give it another try on another computer... $\endgroup$ – RUser4512 Dec 30 '15 at 23:07

I've figured it out in the meantime.

So calling

predict(model1, type='prob')

or model2 for that matter, will evaluate the whole training data. It is the same as

predict(model2,type='prob') == predict(model2$finalModel, iris,type='prob')

Using the predict function on the random Forest model will evaluate the OOB data, so

predict(model2$finalModel, type='prob') == as.data.frame(predict(model3, type='prob'))

Which leads to

predict(model2$finalModel, iris,type='prob') == predict(model3,type='prob')

and this was what was missing when comparing the models results and assessing for reproducibility.

I've left out the call to all.equal in the code above for clarity.


Not the answer you're looking for? Browse other questions tagged or ask your own question.