12
$\begingroup$

Can AUC-ROC values be between 0-0.5? Does the model ever output values between 0 and 0.5?

$\endgroup$
17
$\begingroup$

A perfect predictor gives an AUC-ROC score of 1, a predictor which makes random guesses has an AUC-ROC score of 0.5.

If you get a score of 0 that means the classifier is perfectly incorrect, it is predicting the incorrect choice 100% of the time. If you just changed the prediction of this classifier to the opposite choice then it could predict perfectly and have an AUC-ROC score of 1.

So in practice if you get an AUC-ROC score between 0 and 0.5 you might have a mistake in the way you labeled your classifier targets or you might have a bad training algorithm. If you get a score of 0.2 this shows that the data contains enough information to get a score of 0.8 but something went wrong.

$\endgroup$
  • $\begingroup$ I think this answer skips over the possibility that the model has overfit, for example obtaining AUC of 0.8 on training data but AUC of 0.35 on holdout data. $\endgroup$ – Sycorax Oct 4 '18 at 15:19
  • $\begingroup$ @Sycorax: Hmm, I can see how overfitting could obviously drive the AUC to where it's at chance level (if you're so far from the true model that your predictions are just garbage), but how would it go (significantly) below chance? $\endgroup$ – Ruben van Bergen Feb 4 at 16:20
  • 1
    $\begingroup$ You'll have an AUC below 0.5 whenever the ranking on some set is closer to being backward than correct. It's not any different from overfitting in any other context. $\endgroup$ – Sycorax Feb 4 at 16:30
13
$\begingroup$

They can, if the system you're analyzing performs below chance level. Trivially, you could easily construct a classifier with 0 AUC by having it always answer opposite to the truth.

In practice of course you train your classifier on some data so values very much smaller than 0.5 would typically indicate an error in your algorithm, data labels, or choice of train/test data. E.g. if you mistakenly switched the class labels in your train data your expected AUC would be 1 minus the "true" AUC (given correct labels). The AUC could also be < 0.5 if you split your data into train & test partitions in such a way that the patterns to be classified were systematically different. This might happen (for example) if one class was more common in the train vs. the test set, or if the patterns in each set had systematically different intercepts that you didn't correct for.

Lastly, it could also happen randomly because your classifier is at chance level in the long run but happened to get "unlucky" in your test sample (i.e. get a few more errors than successes). But in that case the values should still be relatively close to 0.5 (how close depends on the number of data points).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.