When performing 5-fold cross-validation (for example), it is typical to compute a separate ROC curve for each of the 5 folds and often times a mean ROC curve with std. dev. shown as curve thickness.
However, for LOO cross-validation, where there is only a single test datapoint in each fold, it doesn't seem sensical to compute a ROC "curve" for this single datapoint.
I have been taking all of my test data points (along with their separately computed p-values) and pooling them into one large set to compute a single ROC curve, but is this the statistically kosher thing to do?
What is the right way to apply ROC analysis when the number of data points in each fold is one (as in the case of LOO cross validation)?