2
$\begingroup$

After training using random forests on the iris dataset, I get an OOB error and a confusion matrix.

data(iris)
(model <- randomForest(Species~., iris, ntree=500,importance=T,do.trace = 100) )
model$oob.times

The help mentions that the confusion matrix is based off the OOB data. But I see the confusion matrix reports values using the same number of samples as used in the training input.

Can anyone explain as to how exactly the confusion matrix reports its error values? Does it use the OOB error value and scale it to the training data-set size, or does it pick samples of OOB data and run the RF again?

$\endgroup$
  • $\begingroup$ OOB is ambiguous here: this is about out-of-bag errors, not out-of-bootstrap (which would be for the non-aggregated models). $\endgroup$ – cbeleites Apr 29 '14 at 7:06
0
$\begingroup$

tl;dr: Try setting do.trace argument to 5 and see how the OOB error reacts.

To answer your first question from documentation

err.rate (classification only) vector error rates of the prediction on the input data, the i-th element being the (OOB) error rate for all trees up to the i-th.

However, your second question gets to the heart of the matter, I think. The algorithm does pick "samples" of the OOB data (it samples the whole data set exactly once!) and run the RF again, this time with a different random set of variables. This is what differentiates a random forest (set of trees) from a single decision tree: it's not the set of data that is random (as in decision tree), it's the set of variables!

model$confusion
# setosa versicolor virginica class.error
# setosa         50          0         0        0.00
# versicolor      0         47         3        0.06
# virginica       0          4        46        0.08

sum(model$confusion[ ,1:3])
#150

nrow(iris)
#150

Thus there is no need to "scale it to the training data set size"- it's at the full size each time it runs through the decision tree.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.