# Cross Validation Random Forest AUC very high

first time poster so hopefully this makes sense and please let me know of any important information I may leave out.

I'm trying to identify my cross-validated AUCs for basic GLM, elastic net, and random forest, using the caret package. See below:

controlroc <- trainControl(method = "repeatedcv",
number = 10,
repeats = 5,
savePredictions = "final",
classProbs = TRUE,
selectionFunction = "oneSE",
summaryFunction = twoClassSummary, allowParallel = TRUE )

set.seed(1234)

model_listworks <- caretList(
DV ~.,
data = train,
trControl=controlroc,
metric="ROC",
tuneList=list(
glmmodel = caretModelSpec(method="glm", family = "binomial"),
enetmodel=caretModelSpec(method="glmnet", tuneGrid = expand.grid (alpha = c (0, .1, .2, .4, .6, .8, 1), lambda = seq (.01, .2, length = 40))),
rfmodel = caretModelSpec(method="rf", tuneGrid = expand.grid (.mtry=c(1:10)), ntrees = 1000)
)
)


As expected, random forest overfitted the training dataset. However, this continues despite CV and I believe CV isn't being accounted for somehow. I have calculated the AUC without CV, shown below with AUC of 0.999, and then swapped the same process with a model that should have CV implemented (i.e., model_listworks) and receive identical AUCs. When I just run the model_listworks model alone, however, I receive the expected CV AUC output of 0.713.

rffit = train(DV ~., data=train, method="rf", tuneGrid = expand.grid (.mtry=c(1:10)), ntrees = 1000, metric="ROC", trControl=controlroc)
print(rffit)

#Train
probsrf1 = predict(rffit, train, type = "prob")
train$$probsrf1 = probsrf1[,"yesi"] roc.rf1 = roc(response = train$$DV, predictor = train$probsrf1) #ROC: 0.9994 probsrf2 = predict(model_listworks$$rfmodel, train, type = "prob") train$$probsrf2 = probsrf2[,"yesi"] roc.rf2 = roc(response = train$$DV, predictor = train$$probsrf2) #ROC: 0.9994 model_listworks$rfmodel
#ROC: 0.7128026


Is there something I'm missing here? I just need my training CV AUCs for these three models.

If you look under model_listworks$rfmodel you will see what is the mtry chosen. In this part of the code: probsrf2 = predict(model_listworks$rfmodel, train, type = "prob")

The training CV AUCs will be under model_listworks, like you have done, model_listworks\$rfmodel for random forest. These are basically the AUC calculated using the test fold.