I am comparing few classifiers and I am slightly confused now. I will call the classifiers A, B, C.
If I draw ROC curve, and estimate AUROC, the result points that the classifier A is the best and the classifier C is the worst.
If I draw dependency of the accuracy on the criteria (position of threshold) and integrate the area under the curve, the classifier A is the far best one.
In case of implementation, I can imagine the classifier that is less dependent on criteria (what can be difficult to set in practice) is better than classifier that scores high in AUROC.
Does some common metric like "area under accuracy on criteria dependency curve" exists, or it is wrong/useless practice?
Is it possible that this area is not corresponding with AUROC?