4
$\begingroup$

From ?combine in randomForest:

The confusion, err.rate, mse and rsq components (as well as the corresponding components in the test compnent, if exist) of the combined object will be NULL.

Why are the err.rate, mse and rsq components for the combined NULL? Is there an efficient way to re-calculate these metrics?

I ask because I would like to figure out a way to use the oob re-sampling method with the parRF model in caret.

$\endgroup$

closed as off-topic by mkt, user158565, mdewey, Sycorax, Michael Chernick Aug 10 at 0:53

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – mkt, user158565, mdewey, Sycorax, Michael Chernick
If this question can be reworded to fit the rules in the help center, please edit the question.

5
$\begingroup$

Because, at least in default setting, the split of objects into bags is lost.

To replicate, you have to train each partial model with keep.inbag=TRUE, then predict the training set with predict(model,trainSet,predict.all=TRUE) and clip it to out-of-bags using bag splits from model$inbag.
Now, you do this for all forests you want to merge, bind and use to calculate the metrics.

(I'll try to extend this to actual code later)

$\endgroup$
  • 8
    $\begingroup$ Is it later yet? =) $\endgroup$ – Zach Feb 6 '12 at 21:08
2
$\begingroup$

According to this answer, the out-of-bag predictions for the combined object can be calculated as follows, as predict.randomForest returns out-of-bag predictions (if newdata is not given):

set.seed(42)
library(randomForest)
rf1 <-randomForest(Species ~ ., iris, ntree=50, norm.votes=FALSE) 
rf2 <- randomForest(Species ~ ., iris, ntree=50, norm.votes=FALSE) 
rf3 <- randomForest(Species ~ ., iris, ntree=50, norm.votes=FALSE) 

rf.all <- combine(rf1, rf2, rf3) 
predict(rf.all, type='prob')
$\endgroup$
0
$\begingroup$

I've posted sample code below which calculates the error rate for the first tree in a classification forecast:

library(randomForest)

# Create dataset
y <- factor(rbinom(300, c(0, 1), 1))
x <- matrix(rnorm(300 * 5), 300, 5, dimnames = list(NULL, paste0("x", 1:5)))
model_df <- data.frame(y, x)

# Estimate model
rf <- randomForest(y ~ ., data = model_df, ntree = 5, keep.forest = TRUE, keep.inbag=TRUE)

# Calculate error on the first tree
pred <- predict(rf,model_df, predict.all=TRUE)
tree_pred_1 <- pred$individual[ , 1]

oob_idx <- rf$inbag[ , 1] == 0

oob_tree_pred_1 <- tree_pred_1[oob_idx]
oob_tree_actual_1 <- rf$y[oob_idx]

conf_mtx <- table(oob_tree_pred_1, oob_tree_actual_1)
error_rates <- 1 - diag(conf_mtx) / colSums(conf_mtx)
error_rate <- 1 - sum(diag(conf_mtx)) / sum(conf_mtx)
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.