1
$\begingroup$

I have precision and recall values and want to measure an estimator performance:

import matplotlib.pyplot as plt
import numpy as np
from sklearn.metrics import precision_recall_curve
from sklearn.metrics import auc, average_precision_score

y_true = np.array([0, 0, 1, 1])
y_scores = np.array([0.1, 0.4, 0.35, 0.8])

precision, recall, threshold = precision_recall_curve(y_true, y_scores)


plt.figure()
plt.plot(recall, precision)
plt.show()

enter image description here

But there is 2 related metrics: sklearn.metrics.auc and sklearn.average_precision_score:

print(auc(recall, precision))
print(average_precision_score(y_true, y_scores))

0.7916666666666666
0.8333333333333333

It seems that average_precision_score would over-estimate AUPRC. So what is the point to use average_precision_score?

UPDATE

I do not agree, that the question is closed. Linked question does not answer on this:

Why AUC under PR curve is less than average_precision score, while one of the arguments against trapezoidal rule is

"This implementation is not interpolated and is different from computing the area under the precision-recall curve with the trapezoidal rule, which uses linear interpolation and can be too optimistic."

So is trapezoidal rule optimistic or pessimistic?

$\endgroup$
0

1 Answer 1

1
$\begingroup$

The average precision directly emphasizes the ability to distinguish between positive and negative examples, whereas area under the precision recall curve (auc) gives an overall measure of performance across all possible classification thresholds.

The average precision score would be more suitable when dealing with imbalanced datasets as it focuses on the positive class being predicted, which is usually the one of interest in an imbalanced scenario. Auc considers both the positive and negative class, which wouldn't helpful if the negative class is significantly larger.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.