I am using the bootstrap approach for internal validation of a multivariate model built with either standard logistic regression OR elastic net.
The procedure I use is as follows:
1) build model using the entire dataset, obtain predicted values, and calculate AUC (AUC_ap, apparent)
2) generate 100-500 bootstrap samples derived from the original dataset
3) for each bootstrap sample, follow the identical procedure as in #1, and obtain predicted values and auc for i) current bootstrap sample, and ii) original dataset
4) calculate difference between i) and ii) (in #3) for each of the 100-500 bootstrap sample, and take the average --> "optimism"
5) calculate optimism-corrected AUC : AUC_ap - optimism
My question is WHAT ROC curve would be best to present in a paper? For example, the ROC derived in step #1 is one choice, but clearly optimistic. Alternatively, I have tried to generate an "average ROC" using the R package ROCR, based on the ROC curves derived in step #3 (ii). However, the AUC for the [average of these ROC curves] I do not believe is equivalent to the value obtained in step #5.
Any input is greatly appreciated! -M