# How is a ROCAUC=1.0 possible with imperfect accuracy? [duplicate]

I used sklearn to compute roc_auc_score for a dataset of 72 instances. The accuracy was at 97% (2 misclassifications), but the ROC AUC score was 1.0. How is this possible? I would think that even one misclassification should have dropped the score to slightly below 1.0.

# Python 3.6.4
# numpy==1.14.3
# scikit-learn==0.19.1
# scipy==1.1.0

from sklearn import metrics
import numpy as np

y_true = np.array([0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0])
y_prob = np.array([0.0, 0.1, 0.0, 0.0, 0.0, 0.0, 0.1, 0.0, 0.1, 0.0, 0.0, 0.0, 0.0, 0.7, 0.5, 0.0, 0.0, 0.0, 0.0, 0.0, 0.1, 0.1, 0.0, 0.0, 0.9, 0.0, 0.0, 0.0, 0.0, 0.1, 0.0, 0.0, 0.0, 0.1, 0.1, 0.0, 0.0, 0.7, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.1, 0.0, 0.0, 0.0, 0.0, 0.1, 0.0, 0.0, 0.1, 0.1, 0.0, 0.9, 0.0, 0.0, 0.4, 0.0, 0.0, 0.0])

# Show which actuals do not match their expected probabilities
for index, (actual, predicted_prob) in enumerate(zip(y_true, y_prob)):
if (actual == 1 and predicted_prob <= 0.5) or (actual == 0 and predicted_prob > 0.5):
print (f'Mismatch at index {index}. Actual={actual}, predicted_prob={predicted_prob}')

rocauc = metrics.roc_auc_score(y_true, y_prob)
print (f'ROCAUC: {rocauc}')

# Outputs:
# Mismatch at index 14. Actual=1.0, predicted_prob=0.5
# Mismatch at index 68. Actual=1.0, predicted_prob=0.4
# ROCAUC: 1.0


Then I debugged the score computation itself and looked at the coordinate ROC output.

# In sklearn/metrics/ranking.py, line 271:
--> 271         fpr, tpr, tresholds = roc_curve(y_true, y_score,
272                                         sample_weight=sample_weight)

ipdb> fpr
array([0.        , 0.        , 0.        , 0.18181818, 1.        ])
ipdb> tpr
array([0.33333333, 0.66666667, 1.        , 1.        , 1.        ])
ipdb> tresholds
array([0.9, 0.7, 0.4, 0.1, 0. ])

# coords = []
# for x, y in zip(fpr, tpr):
#   coords.append((x, y))

ipdb> pp coords
[(0.0, 0.3333333333333333),
(0.0, 0.6666666666666666),
(0.0, 1.0),
(0.18181818181818182, 1.0),
(1.0, 1.0)]


All of those coordinates are on x=0 or y=1, meaning ROCAUC is showing 1.0. The only plausible explanation I can think of is that if the alg had more fpr/tpr points, it would show a very tight curve that never reaches x,y = (0, 1), and the ROCAUC would be close to 1, but not exactly 1. Is that a reasonable interpretation or am I missing something?

## marked as duplicate by Sycorax, kjetil b halvorsen, mdewey, Peter Flom♦Aug 25 '18 at 11:53

ROC AUC and the $c$-statistic are equivalent, and measure the probability that a randomly-chosen positive sample is ranked higher than a randomly-chosen negative sample. If all positives have score 0.49 and all negatives have score 0.48, then the ROC AUC is 1.0 because of this property. This can lead to counter-intuitive results. In this hypothetical, the accuracy, using the rule of a 0.5 cutoff, is 0.0 because all of the predictions are below 0.5!