The Area Under ROC curve (AUROC) is a quantity used to quantify performance of classifiers. I am currently interested in the most basic drawbacks of using AUROC as unique performance measure.
I got a negative feedback with AUROC in some recent analysis as I met a classification problem performed through a logistic regression in presence of complete separation; I presume that in this case one simply obtains a perfect AUROC=1. Is this correct?
In general, extremely high values of AUROC (I mean values $\geq 0.998$ ) would make me wonder whether there is something "wrong" or "artifical" with the model and/or data, as first feeling.
It would be nice if you could confirm this feeling of mine and support it with examples. At this stage, the opposite thesis is also interesting to me.