22
$\begingroup$

Is Average Precision (AP) the Area under Precision-Recall Curve (AUC of PR-curve) ?

EDIT:

here is some comment about difference in PR AUC and AP.

The AUC is obtained by trapezoidal interpolation of the precision. An alternative and usually almost equivalent metric is the Average Precision (AP), returned as info.ap. This is the average of the precision obtained every time a new positive sample is recalled. It is the same as the AUC if precision is interpolated by constant segments and is the definition used by TREC most often.

http://www.vlfeat.org/overview/plots-rank.html

Moreover, the auc and the average_precision_score results are not the same in scikit-learn. This is strange, because in the documentation we have:

Compute average precision (AP) from prediction scores This score corresponds to the area under the precision-recall curve.

here is the code:

# Compute Precision-Recall and plot curve
precision, recall, thresholds = precision_recall_curve(y_test, clf.predict_proba(X_test)[:,1])
area = auc(recall, precision)
print "Area Under PR Curve(AP): %0.2f" % area  #should be same as AP?

print 'AP', average_precision_score(y_test, y_pred, average='weighted')
print 'AP', average_precision_score(y_test, y_pred, average='macro')
print 'AP', average_precision_score(y_test, y_pred, average='micro')
print 'AP', average_precision_score(y_test, y_pred, average='samples')

for my classifer I have something like:

Area Under PR Curve(AP): 0.65
AP 0.676101781304
AP 0.676101781304
AP 0.676101781304
AP 0.676101781304
$\endgroup$
13
$\begingroup$

Short answer is: YES. Average Precision is a single number used to summarise a Precision-Recall curve:

enter image description here

You can approximate the integral (area under the curve) with:

enter image description here

Please take a look at this link for a good explanation.

$\endgroup$
  • $\begingroup$ What about this comment? "The AUC is obtained by trapezoidal interpolation of the precision. An alternative and usually almost equivalent metric is the Average Precision (AP), returned as info.ap. This is the average of the precision obtained every time a new positive sample is recalled. It is the same as the AUC if precision is interpolated by constant segments and is the definition used by TREC most often." vlfeat.org/overview/plots-rank.html $\endgroup$ – mrgloom Jun 15 '15 at 10:44
  • 1
    $\begingroup$ I think the average of the precision obtained every time a new positive sample is recalled refers to Interpolated average precision explained in the link I gave. Some authors choose an alternate approximation that is called the interpolated average precision. Confusingly, they still call it average precision. $\endgroup$ – Zhubarb Jun 15 '15 at 10:50
  • $\begingroup$ a few quick questions: 1) why coordinate (recall=0, precision=1)? makes no sense to me at all. 2) as you might observe, if we lower the classifier's threshold, more results could be returned, and as a consequence, recall might not increase, but precision could vary, for example, with 2 positive items in total, here is the ranked results = [False, True, False, False, True], so p-r pairs = [(p=0, r=0), (1/2, 1/2), (1/3, 1/2), (1/4, 1/2), (2/5, 2/2)], as you can see, for r=1/2, there're 3 p (i.e. 1/2, 1/3, 1/4), just like in your graph at r=0.8, it's ok just plot them at the same x axis? $\endgroup$ – avocado Dec 23 '17 at 9:43
2
$\begingroup$

average_precision_score function expect confidence or probability as second parameter.

so you should use it as below,

average_precision_score(y_test, clf.predict_proba(X_test)[:,1])

and then it's same result of auc function.

$\endgroup$
  • $\begingroup$ Different examples in WEKA software and scikit-learn provide CLF score but not AUC. Can it be that this CLF score is actually somehow related to AUC or even be AUC? $\endgroup$ – hhh Jul 7 '17 at 0:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.