# In R, why do they have so different performance between RF models with reduced feature vs ones built manually?

For a regression problem, I'm comparing two randomforest (RF) models:

(a) A resulted model after doing feature selection by caret::rfe().

As I know, the recursive feature elimination (RFE) algorithm automatically build a tuned model with the optimal feature subset. This resulted model can be used directly for prediction.

(b) A RF model built by randomForest::randomForest() using the same parameter value and feature subset with those of model (a).

Above two RF models were used to predict numeric data. I used airquality dataset in R to create an example code.

I only used numeric columns of the dataset and this set was split by 10-fold CV method (train:test=9:1).

The R2 of (a) was about 0.94 whereas that of (b) was about 0.52.
I don't expect that the models show the same performance, but I at least expected similar one.

Why these models have so different results?

Thank you

# load library

library(caret)
library(randomForest)
library(dplyr)

# dataset
data =
airquality[complete.cases(airquality), ] %>%
select(Ozone:Temp)

# run RFE

set.seed(100)

index_train = createMultiFolds(data$Ozone, times = 5, k = 10) pred_vars = names(data)[!(names(data) %in% c("Ozone"))] ctrl_rf = rfeControl(method = "repeatedcv", repeats = 5, verbose = TRUE, functions = rfFuncs, index = index_train) rf_rfe = rfe(x = data[, pred_vars], y = data$Ozone,
size = c(1:length(pred_vars)),
rfeControl = ctrl_rf,
ntree = 1000)

# build RF manually

rf_rfe$$fit rf_var_selectd = rf_rfe$$optVariables

train_idx = sample(1:dim(data)[1], dim(data)[1] * 0.1) # 10-fold cv
rf_manual = randomForest(Ozone ~ .,
data = data[, c("Ozone", rf_var_selectd)],
subset = train_idx,
mtry = 1, # default: p/3 = 6
ntree = 1000,
importance = TRUE)

# predict
rf_rfe_test = predict(rf_rfe,
newdata = data[-train_idx, ])
rf_manual_test = predict(rf_manual,
newdata = data[-train_idx, ])

summary(lm(rf_rfe_test ~ data[-train_idx, "Ozone"])) # R2 0.94
summary(lm(rf_manual_test ~ data[-train_idx, "Ozone"])) # R2 0.52