1
$\begingroup$

I am using R package 'randomForest' and have noticed that when I try to make predictions with a fitted randomForest object and pass the data used to fit the model as the "new data", I get back exactly the response values, despite the confusion matrix for the fitted model not being diagonal. Here is an example:

set.seed(1234)
x1 <- rnorm(200) 
x2 <- rnorm(200) 
y <- x1-x2>0
D <- data.frame( cbind(y,x1,x2) ) 
D$y <- as.factor(D$y)

model <- randomForest(y~., data=D)
model

Call:
 randomForest(formula = y ~ ., data = D) 
           Type of random forest: classification
                 Number of trees: 500
 No. of variables tried at each split: 1

    OOB estimate of  error rate: 4%
Confusion matrix:
    0  1 class.error
0 111  5  0.04310345
1   3 81  0.03571429

Note the non-diagonal confusion matrix. Now, when I pass the original data to the "predict" function, I get perfect agreement, which is inconsistent with the confusion matrix.

p <- predict(model,D)
sum( p != D$y ) 
[1] 0

Is this a property of the model, or a misunderstanding on my part of what the program is doing? I rather doubt the former, because when I used "predict", without passing the data (which, I assume, gives the in-sample predictions), I get

p1 <- predict(model)
sum( p1 != D$y ) 
[1] 8

which gives me 8 disagreements, which concurs with the confusion matrix. What's going on here?

$\endgroup$
3
$\begingroup$

When you call predict(model) this return the out of bag predicitions performed by the random forest.

However, when you call predict(model, training_data) the random forest applies its prediction to the training set, leading to a perfect accuracy (unless you specified an early stopping criterion on the growth of the trees)

$\endgroup$
  • $\begingroup$ Thanks! What are "out of bag predictions"? Predictions made on the test data? $\endgroup$ – HammyD Sep 14 '15 at 15:23
  • $\begingroup$ When you run a random forest, each tree is train on a random subset of the data (usually, around 60% of the lines and sqrt(nb of predictors) predictors (for classification)). Out of bag predictions are predictions on the remaining lines. $\endgroup$ – RUser4512 Sep 14 '15 at 16:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.