Could someone please explain me the answer of the question already given here:
Combining randomForests in R, why are the err.rate, mse and rsq components NULL
I do not understand the phrase "clip it to out-of-bag" from model$inbag.
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this communityCould someone please explain me the answer of the question already given here:
Combining randomForests in R, why are the err.rate, mse and rsq components NULL
I do not understand the phrase "clip it to out-of-bag" from model$inbag.
Each randomForest model consists of decision trees. The number of trees in randomForest()
function is controlled by the parameter ntree
. Each tree is built using not all observation, but a sample with replacement. This approach is called "bagging". model$inbag
is a n x ntree matrix, such that the element model$inbag[i,j]
is 1 if the observation i have been used to train j tree and else 0. Prediction of response i by tree j when model$inbag[i,j]
= 0 is called out-of-bag. The error rate, mse and r-squared usually are derived from out-of-bag predictions, and thus are unbiased. By default, predict()
function combines both in-bag and out-of-bag predictions to output single decision. We need to separate out-of-bag predictions from in-bag. Thus we use predict.all=TRUE
as an additional argument, and out-of-bag predictions can be accessed through prediction_output$individual[!model$inbag]
. Such outputs from several combined models should be merged together, and the ensemble prediction value for each i should be obtained as the most-voted class (in case of classification) or mean (in case of regression). Then we can calculate combined model statistics using this prediction values and the actual ones.
You can read the basis of random forest from official site http://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm
predict()
incorrectly. You can invoke the help on it entering ?predict.randomForest
. If we don't pass the parameter newdata
, the out-of-bag prediction is returned. Taking your code, just compare: table(predict(rf1),predict(rf1,iris[,1:4]))
. I have checked this option in case of combined model - this feature is still present. So really you don't need to care about bagging, but calculate your statistics directly: if classification - err.rate<- 1-sum(diag(table(predict(rf.all), rf.all$y)))/sum(table(predict(rf.all), rf.all$y))
else if regression - mse<-mean((predict(rf.all)- rf.all$y)^2); r.squared<-1-sum((predict(rf.all)- rf.all$y)^2)/sum((mean(rf.all$y)- rf.all$y)^2)
.
Here's a function to generate the confusion matrix after combining trees (parallel execution, etc):
getConfusionMatrix <- function(rf) {
tbl = table(predict(rf), rf$y)
class.error = vector()
for (i in 1:nrow(tbl)) {
rowSum = sum(tbl[i,])
accurate = diag(tbl)[i]
error = rowSum - accurate
class.error[i] = error / rowSum
}
return(cbind(tbl, class.error))
}
To use it on the code above:
rf.all$confusion = getConfusionMatrix(rf.all)