Relation of criteria selection and AUROC I am comparing few classifiers and I am slightly confused now. I will call the classifiers A, B, C.
If I draw ROC curve, and estimate AUROC, the result points that the classifier A is the best and the classifier C is the worst.
If I draw dependency of the accuracy on the criteria (position of threshold) and integrate the area under the curve, the classifier A is the far best one.
In case of implementation, I can imagine the classifier that is less dependent on criteria (what can be difficult to set in practice) is better than classifier that scores high in AUROC.
My questions:


*

*Does some common metric like "area under accuracy on criteria dependency curve" exists, or it is wrong/useless practice?

*Is it possible that this area is not corresponding with AUROC?
 A: This answer assumes a balanced dataset, with as many positive and negative observations. I believe it is enough to show my point.
What does ROC analysis teach us? Let's take a plot of the ROC space:

ROC analysis tells us that any classifier can be represented by a monotone curve, that goes from the bottom left (when threshold is high and everything is classified as negative) to the top right (when threshold is low and everything is classified as positive).
What about the accuracy? It is maximal at the top left corner (perfect classification) and minimal at the bottom right (100% wrong classification). If your classifier performs no better than random, the ROC curve will follow the diagonal line, and the accuracy will be 50% regardless of the threshold.
Again: all classifiers will need to go from the bottom left to the top right corner, both having 50% accuracy. How can we have a classifier whose accuracy doesn't depend on the threshold?
Easy: follow the diagonal line with a classifier that performs no better than random.
In other words, the accuracy of a useful classifier has to depend on the threshold. So your following statement is incorrect:

In case of implementation, I can imagine the classifier that is less
  dependent on criteria (what can be difficult to set in practice) is
  better than classifier that scores high in AUROC.

and therefore an "area under accuracy on criteria dependency curve" would be useless.
