I am comparing few classifiers and I am slightly confused now. I will call the classifiers A, B, C.
If I draw ROC curve, and estimate AUROC, the result points that the classifier A is the best and the classifier C is the worst.
If I draw dependency of the accuracy on the criteria (position of threshold) and integrate the area under the curve, the classifier A is the far best one.
In case of implementation, I can imagine the classifier that is less dependent on criteria (what can be difficult to set in practice) is better than classifier that scores high in AUROC.
My questions:
Does some common metric like "area under accuracy on criteria dependency curve" exists, or it is wrong/useless practice?
Is it possible that this area is not corresponding with AUROC?