# Why do I need bag composition to calculate OOB error of combined random forest model?

Combining randomForests in R, why are the err.rate, mse and rsq components NULL

I do not understand the phrase "clip it to out-of-bag" from model$inbag. - Next time, please just use a comment to ask clarification about some answer. – mbq Sep 3 '12 at 17:05 data(iris) rf1 <- randomForest(Species ~ ., iris, ntree=50, norm.votes=FALSE) rf2 <- randomForest(Species ~ ., iris, ntree=50, norm.votes=FALSE) rf3 <- randomForest(Species ~ ., iris, ntree=50, norm.votes=FALSE) rf.all <- combine(rf1, rf2, rf3) print(rf.all) – Z Khan Sep 4 '12 at 13:56 Ok, well, I meant "please add comment under post that troubles you instead of asking a new question linking it". If you want to clarify your post, please use edit option. – mbq Sep 4 '12 at 14:39 How can I merge outputs from combined models and how to calculate mse, rsq and error rate. Can you please post some sample code to do it for two models each with 10 trees – Z Khan Sep 4 '12 at 14:41 add comment ## 2 Answers Each randomForest model consists of decision trees. The number of trees in randomForest() function is controlled by the parameter ntree. Each tree is built using not all observation, but a sample with replacement. This approach is called "bagging". model$inbag is a n x ntree matrix, such that the element model$inbag[i,j] is 1 if the observation i have been used to train j tree and else 0. Prediction of response i by tree j when model$inbag[i,j] = 0 is called out-of-bag. The error rate, mse and r-squared usually are derived from out-of-bag predictions, and thus are unbiased. By default, predict() function combines both in-bag and out-of-bag predictions to output single decision. We need to separate out-of-bag predictions from in-bag. Thus we use predict.all=TRUE as an additional argument, and out-of-bag predictions can be accessed through prediction_output$individual[!model$inbag]. Such outputs from several combined models should be merged together, and the ensemble prediction value for each i should be obtained as the most-voted class (in case of classification) or mean (in case of regression). Then we can calculate combined model statistics using this prediction values and the actual ones. You can read the basis of random forest from official site http://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm

I have to apologise, I described the default behavior of predict() incorrectly. You can invoke the help on it entering ?predict.randomForest. If we don't pass the parameter newdata, the out-of-bag prediction is returned. Taking your code, just compare: table(predict(rf1),predict(rf1,iris[,1:4])). I have checked this option in case of combined model - this feature is still present. So really you don't need to care about bagging, but calculate your statistics directly: if classification - err.rate<- 1-sum(diag(table(predict(rf.all), rf.all$y)))/sum(table(predict(rf.all), rf.all$y)) else if regression - mse<-mean((predict(rf.all)- rf.all$y)^2); r.squared<-1-sum((predict(rf.all)- rf.all$y)^2)/sum((mean(rf.all$y)- rf.all$y)^2).

-
How can I merge outputs from combined models and how to calculate mse, rsq and error rate. Can you please post some sample code to do it for two models each with 10 trees. –  Z Khan Sep 4 '12 at 14:00
Also it is worth to mention that building several random forests with the same algorithm parameters and then combining models into ensemble is equal to building one random forest but with increased number of trees. –  O_Devinyak Sep 5 '12 at 12:25
Thank you very much! It solved my problem. –  Z Khan Sep 5 '12 at 14:16
Hi Fosgen, I need to calculate cross validated (10-fold for example) error rate for the same combined object of random forest if you could please help me. –  Z Khan Sep 18 '12 at 16:41
getConfusionMatrix <- function(rf) {
tbl = table(predict(rf), rf$y) class.error = vector() for (i in 1:nrow(tbl)) { rowSum = sum(tbl[i,]) accurate = diag(tbl)[i] error = rowSum - accurate class.error[i] = error / rowSum } return(cbind(tbl, class.error)) }  To use it on the code above: rf.all$confusion = getConfusionMatrix(rf.all)