Can AUC-ROC be between 0-0.5?

Question

Can AUC-ROC values be between 0-0.5? Does the model ever output values between 0 and 0.5?

Answer 1

A perfect predictor gives an AUC-ROC score of 1, a predictor which makes random guesses has an AUC-ROC score of 0.5.

If you get a score of 0 that means the classifier is perfectly incorrect, it is predicting the incorrect choice 100% of the time. If you just changed the prediction of this classifier to the opposite choice then it could predict perfectly and have an AUC-ROC score of 1.

So in practice if you get an AUC-ROC score between 0 and 0.5 you might have a mistake in the way you labeled your classifier targets or you might have a bad training algorithm. If you get a score of 0.2 this shows that the data contains enough information to get a score of 0.8 but something went wrong.

Answer 2

I am sorry, but these answers are dangerously wrong. No, you cannot just flip AUC after you see the data. Imagine you are buying stocks, and you always bought the wrong one, but you said to yourself, then it's ok, because if you were purchasing the opposite of what your model was predicting, then you would make money.

The thing is that there are many, often non-obvious reasons how you can bias your results and get consistently below-average performance. If you now flip your AUC, you might think you are the best modeler in the world, although there was never any signal in the data.

Here is a simulation example. Notice that the predictor is just a random variable with no relationship to the target. Also, notice that the average AUC is around 0.3.

library(MLmetrics)
aucs <- list()
for (sim in seq_len(100)){
  n <- 100
  df <- data.frame(x=rnorm(n),
               y=c(rep(0, n/2), rep(1, n/2)))
  
  predictions <- list()
  for(i in seq_len(n)){
    train <- df[-i,]
    test <- df[i,]
    
    glm_fit <- glm(y ~ x, family = 'binomial', data = train)
    predictions[[i]] <- predict(glm_fit, newdata = test, type = 'response')
  }
  predictions <- unlist(predictions)
  aucs[[sim]] <- MLmetrics::AUC(predictions, df$y)
}
aucs <- unlist(aucs)
plot(aucs); abline(h=mean(aucs), col='red')

Results

Of course, there is no way a classifier could learn anything from the data since the data are random. The bellow chance AUC is there because LOOCV creates a biased, unbalanced training set. However, that doesn't mean that if you don't use LOOCV, you are safe. The point of this story is that there are ways, many ways how the results can have bellow average performance even if there is nothing in the data, and therefore you should not flip the predictions unless you know what you are doing. And since you've got bellow average performance, you don't see what you are doing :)

Here is a couple of papers that touched this problem, but I am sure others did as well

Classification based hypothesis testing in neuroscience: Below‐chance level classification rates and overlooked statistical properties of linear parametric classifiers by Jamalabadi et al (2016).

How to control for confounds in decoding analyses of neuroimaging data by Snoek et al (2019).

Answer 3

They can, if the system you're analyzing performs below chance level. Trivially, you could easily construct a classifier with 0 AUC by having it always answer opposite to the truth.

In practice of course you train your classifier on some data so values very much smaller than 0.5 would typically indicate an error in your algorithm, data labels, or choice of train/test data. E.g. if you mistakenly switched the class labels in your train data your expected AUC would be 1 minus the "true" AUC (given correct labels). The AUC could also be < 0.5 if you split your data into train & test partitions in such a way that the patterns to be classified were systematically different. This might happen (for example) if one class was more common in the train vs. the test set, or if the patterns in each set had systematically different intercepts that you didn't correct for.

Lastly, it could also happen randomly because your classifier is at chance level in the long run but happened to get "unlucky" in your test sample (i.e. get a few more errors than successes). But in that case the values should still be relatively close to 0.5 (how close depends on the number of data points).