Bagged Decision trees / Random Forests: why ISLR uses validation set instead of OOB to compute out-of-sample MSE?

Question

I am reading the book "An Introduction to Statistical Learning" available here.

Chapter 8.3.3 at page 328 of the book computes a bagged decision tree (which is a random forest where we use all of the predictors at each step). It does so for a regression problem. It then calculates out-of-sample MSE using a separate test set.

The code goes like that:

library(MASS)
library(randomForest)
set.seed(1)

train = sample(1:nrow(Boston), nrow(Boston)/2)

# Bagging = Bootstrap aggregation
bag.boston = randomForest(medv~.,
                          data=Boston,
                          subset=train,
                          mtry=13,
                          importance=T)

yhat <- predict(bag.boston,
                newdata=Boston[-train,])
y <-Boston[-train,]$medv
mse = mean((y-yhat)^2)
print(paste0("MSE: ", mse))

To be truth, at each step the training set is bootstrapped, and the decision tree is trainined on the bootstrapped dataset. Some samples are not drawn during bootstrapping, and are used after training to compute MSE. At least this is how I understood it.

Why the book uses a separate validation set, instead of OOB samples, to compute out-of-sample MSE?

Said otherwise, why the following is wrong?

print(mean(bag.boston$mse))

I have tried it, and the results are as follows. The code:

mse = mean((y-yhat)^2)
print(paste0("MSE: ", mse))

returns 19.88, while the code

print(mean(bag.boston$mse))

returns a much lower 11.72. So it seems that the latter is overfitted, but I don't understand why.

Answer 1

By a simulation experiment, I discovered that when the sample size is large, the performance metrics calculated in the two ways (OOB and validation set) are numerically very very similar (the maximum difference is 0.1).

I have tried with accuracy, sensitivity, specificity and F1.