Could someone please explain me the answer of the question already given here:
Combining randomForests in R, why are the err.rate, mse and rsq components NULL
I do not understand the phrase "clip it to out-of-bag" from model$inbag.
$\begingroup$
$\endgroup$
4
-
2$\begingroup$ Next time, please just use a comment to ask clarification about some answer. $\endgroup$– user88Sep 3, 2012 at 17:05
-
$\begingroup$ data(iris) rf1 <- randomForest(Species ~ ., iris, ntree=50, norm.votes=FALSE) rf2 <- randomForest(Species ~ ., iris, ntree=50, norm.votes=FALSE) rf3 <- randomForest(Species ~ ., iris, ntree=50, norm.votes=FALSE) rf.all <- combine(rf1, rf2, rf3) print(rf.all) $\endgroup$– Z KhanSep 4, 2012 at 13:56
-
$\begingroup$ Ok, well, I meant "please add comment under post that troubles you instead of asking a new question linking it". If you want to clarify your post, please use edit option. $\endgroup$– user88Sep 4, 2012 at 14:39
-
$\begingroup$ How can I merge outputs from combined models and how to calculate mse, rsq and error rate. Can you please post some sample code to do it for two models each with 10 trees $\endgroup$– Z KhanSep 4, 2012 at 14:41
Add a comment
|
2 Answers
$\begingroup$
I have to apologise, I described the default behavior of
$\endgroup$
3
Each randomForest model consists of decision trees. The number of trees in randomForest() function is controlled by the parameter ntree. Each tree is built using not all observation, but a sample with replacement. This approach is called "bagging". model$inbag is a n x ntree matrix, such that the element model$inbag[i,j] is 1 if the observation i have been used to train j tree and else 0. Prediction of response i by tree j when model$inbag[i,j] = 0 is called out-of-bag. The error rate, mse and r-squared usually are derived from out-of-bag predictions, and thus are unbiased. By default, predict() function combines both in-bag and out-of-bag predictions to output single decision. We need to separate out-of-bag predictions from in-bag. Thus we use predict.all=TRUE as an additional argument, and out-of-bag predictions can be accessed through prediction_output$individual[!model$inbag]. Such outputs from several combined models should be merged together, and the ensemble prediction value for each i should be obtained as the most-voted class (in case of classification) or mean (in case of regression). Then we can calculate combined model statistics using this prediction values and the actual ones.
You can read the basis of random forest from official site http://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm
I have to apologise, I described the default behavior of
predict() incorrectly. You can invoke the help on it entering ?predict.randomForest. If we don't pass the parameter newdata, the out-of-bag prediction is returned. Taking your code, just compare: table(predict(rf1),predict(rf1,iris[,1:4])). I have checked this option in case of combined model - this feature is still present. So really you don't need to care about bagging, but calculate your statistics directly: if classification - err.rate<- 1-sum(diag(table(predict(rf.all), rf.all$y)))/sum(table(predict(rf.all), rf.all$y)) else if regression - mse<-mean((predict(rf.all)- rf.all$y)^2); r.squared<-1-sum((predict(rf.all)- rf.all$y)^2)/sum((mean(rf.all$y)- rf.all$y)^2).
-
$\begingroup$ How can I merge outputs from combined models and how to calculate mse, rsq and error rate. Can you please post some sample code to do it for two models each with 10 trees. $\endgroup$– Z KhanSep 4, 2012 at 14:00
-
$\begingroup$ Also it is worth to mention that building several random forests with the same algorithm parameters and then combining models into ensemble is equal to building one random forest but with increased number of trees. $\endgroup$ Sep 5, 2012 at 12:25
-
$\begingroup$ Hi Fosgen, I need to calculate cross validated (10-fold for example) error rate for the same combined object of random forest if you could please help me. $\endgroup$– Z KhanSep 18, 2012 at 16:41
$\begingroup$
$\endgroup$
Here's a function to generate the confusion matrix after combining trees (parallel execution, etc):
getConfusionMatrix <- function(rf) {
tbl = table(predict(rf), rf$y)
class.error = vector()
for (i in 1:nrow(tbl)) {
rowSum = sum(tbl[i,])
accurate = diag(tbl)[i]
error = rowSum - accurate
class.error[i] = error / rowSum
}
return(cbind(tbl, class.error))
}
To use it on the code above:
rf.all$confusion = getConfusionMatrix(rf.all)
