This seems like a basic questions, so I'm likely missing the big picture here...
I would like to know the AUC of each fold of a cross-validation performed in Caret's train function, in order to calculate the standard error of the AUC. Here is the typical output, which I assume provides the average "ROC" (AUC) across each of the five folds:
Generalized Linear Model 85537 samples 31 predictor 2 classes: 'X0', 'X1' No pre-processing Resampling: Cross-Validated (5 fold) Summary of sample sizes: 68430, 68429, 68430, 68430, 68429 Resampling results: ROC Sens Spec 0.918912 0.834479 0.8450047
Other functions (e.g., cv.glmnet in the glmnet package) seems to output the standard error.
Call: cv.glmnet(x = train.data.x, y = train.data.y, type.measure = c("auc"), nfolds = 5, alpha = 1, family = "binomial") Measure: AUC Lambda Measure SE Nonzero min 0.0000504 0.9027 0.0006507 166 1se 0.0003901 0.9021 0.0006224 160
Is there a statistical reason why Caret's train function does not output this as a default? I believe this would help to show the variation in the model performance for different models (e.g., logistic regression and Lasso).
I am likely operating under some false assumptions here.