I'm evaluating a grid of tuning parameters using Caret with metric="ROC" for cross-validation. Is there any simple way to use as metric the area under the curve for an specified interval of the ROC curve?
My code is similar to this:
fitControl <- trainControl(method = "repeatedcv", number = 10, repeats = 10, classProbs = TRUE, ## Evaluate performance using ## the following function summaryFunction = twoClassSummary) gbmFit3 <- train(Class ~ ., data = training, method = "nnet", trControl = fitControl, verbose = FALSE, tuneGrid = myGrid, metric = "ROC")
And I would like, using caret, a metric as the partial area under the curve. The most simple way I think it's using cross-validation + pROC package without using caret, but I would like to know if there is a simple way to do it before I try my custom cv.