Using partial AUC as Caret metric for cross-validation?

Question

I'm evaluating a grid of tuning parameters using Caret with metric="ROC" for cross-validation. Is there any simple way to use as metric the area under the curve for an specified interval of the ROC curve?

My code is similar to this:

fitControl <- trainControl(method = "repeatedcv",
                       number = 10,
                       repeats = 10,
                       classProbs = TRUE,
                       ## Evaluate performance using 
                       ## the following function
                       summaryFunction = twoClassSummary)

gbmFit3 <- train(Class ~ ., data = training,
             method = "nnet",
             trControl = fitControl,
             verbose = FALSE,
             tuneGrid = myGrid,
             metric = "ROC")

And I would like, using caret, a metric as the partial area under the curve. The most simple way I think it's using cross-validation + pROC package without using caret, but I would like to know if there is a simple way to do it before I try my custom cv.

Anyone?

Answer 1

You can emulate what the package's twoClassSummary function does. See the help page for custom performance metrics.

Max