I am trying to understand how does building a regression model with caret's train () function differs from randomForest().

For my excercise, I am using the iris dataset. As shown in the code below, I have tried to replicate in caret the arguments that I can control in randomForest. The task has proven difficult because:

  1. The train() function does not have a metric = "mse". Instead of it, a "RMSE", but, is it an average of the RMSE calculated for each tree on the oob data? or the median?
  2. The mse value returned by randomForest() after print(model.rf) corresponds to the mse of the last trained tree! Why? Or, is it pure coincidence and in reality the print(model.rf) returns a statistic of the mse's?
  3. As I understand, if sampsize argument from randomForest() is not defined AND resample = TRUE, then 0.632 of the cases are resampled with replacement. Is there an argument in train() that is similar to sampsize, or what is the default used?

I would really appreciate a hint on the questions that I have and/or the way I am trying to compare both functions. Thank you!

Below the code:

library(caret)
library(randomForest)
data(iris)

my_iris <- iris[-5]

random forest model with randomForest()

set.seed(1)
model.rf <- randomForest(x = my_iris[,-1], y = my_iris[,1], 
                         ntree = 500,mtry = ncol(my_iris[,-1])/3,    replace = TRUE, 
                         nodesize = 5, maxnodes = NULL, nPerm = 1)


print(model.rf)
Mean of squared residuals: ***0.1245198***

# To try to understand this mse I looked at:

tail(model.rf$mse) # coincides with the returned result
[1] 0.1247448 0.1247001 0.1245805 0.1245146 0.1245797 ***0.1245198***

mean(model.rf$mse) #differs from the returned result
[1] 0.1241558

median(model.rf$mse) # differs from the returned result
[1] 0.1231984

# To compare with the results from caret:

sqrt(tail(model.rf$mse, n=1))
[1] 0.3528736

mean(sqrt(model.rf$mse))
[1] 0.3522906

median(sqrt(model.rf$mse))
[1] 0.3509963

random forest model with caret's train()

set.seed(1)
model.rfc <- train(x = my_iris[,-1], y = my_iris[,1],
                                 method = "rf", 
                                 preProcess=NULL, 
                                 metric = "RMSE",
                                 replace = TRUE,
                                 tuneLength = 1,
                                 ntree = 500,
                                 trControl = trainControl(method = "oob"))

print(model.rfc)
RMSE = 0.3507244

# The RMSE from train() does not match with any of the RMSE returned by randomForest

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.