When we use cross validation (for example 10 fold) I can obtain 10 sensitivity, specificity and accuracy measures and also 10 ROC curves with the associated AUC.
What should I do for reporting my results? Should I report the result of my best model or the average if all of them?
Accuracy metrics of cross-validation are difficult because it lives in a statistical gray zone where the samples aren't coming from the same model, but the models aren't independent either since most of the training data are the same. Ideally, you should avoid the situation all together by setting aside an explicit testing set. If that is not possible (ex. the dataset is too small to set any aside and still do CV), it is generally better to avoid aggregation and report the statistics for each fold. This can tell you a lot about the stability of the model because an average accuracy of 80% where half the folds have an accuracy of 100% and half have an accuracy of 60% has a very different interpretation than one where every fold is 80%.
You can additionally report the aggregate measures, but make it clear what you are doing. Putting the aggregate along side a plot of the individual folds usually does this well.
I report the mean and standard deviation of fold statistics. Sometimes, I also aggregate by calculating the metrics on predicted vs actual ignoring the folds. For instance, if you are looking at RMSE, then pull all predictions and actuals into one set, and calculate RMSE on it pretending there were no folds.
