I'm testing several classifiers in Weka Experimenter. Some of them have — at the same time — low accuracy (Percent_correct statistic) and high AUC. How should the quality of such classifiers be interpreted? Should they be considedered bad (for their low accuracy) or good (for their high AUC)? Under which circumstances one or the other of these performance measures should prevail in judging quality?
Note: Both questions mentioned in the comments below add useful insights. However, I would also like to know when you want to strieve for accuracy (possibly at the cost of worse AUC) and when you want to strieve for better AUC (possibly at the cost of worse accuracy).
