10
$\begingroup$

Wikipedia says that an interpretation of the area under the ROC curve is: "the area under the curve is equal to the probability that a classifier will rank a randomly chosen positive instance higher than a randomly chosen negative one".

But is it the same interpretation of the area under the PR curve? If not, can you please give me an intuitive interpretation for it like the above?

Edit: PR == Precision-Recall

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ Because I wasn't sure exactly what you meant by "PR curve" (the world has too many acronyms to be confident of any of them), I googled it and one of the top hits is a presentation answering your question $\endgroup$
    – whuber ♦
    Commented Oct 16, 2014 at 23:19
  • $\begingroup$ @whuber see edit. $\endgroup$
    – Jack Twain
    Commented Oct 17, 2014 at 7:50

2 Answers 2

Reset to default
1
$\begingroup$

The area under the PR-Curve is ill-defined. Because there is no well-defined precision at recall 0: you get a division by zero there.

You also cannot close this gap easily - it may be anything from 0 to 1, depending on how well your retrieval works.

There is a common approximation to this - AveP, average precision.

Improve this answer
$\endgroup$
3
  • $\begingroup$ When recall is 0, not necessarily, you get a division by zero. A recall = 0 => True Positives = 0. However, the classifier might predict many cases as positives without hiting a true one. In such case, False Positives is defined and precision will be 0 also. What do you think? $\endgroup$
    – soufanom
    Commented Oct 17, 2014 at 20:15
  • $\begingroup$ If you do this, the perfect method will not have a score of 1 either. IMHO, this is not the proper way to resolve this. Just stick to AveP. $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Oct 18, 2014 at 11:09
  • $\begingroup$ @Anony-Mousse can you please see my question here: stats.stackexchange.com/questions/123697/… $\endgroup$
    – Jack Twain
    Commented Nov 12, 2014 at 17:34
0
$\begingroup$

Well, I will try to give some intuition close to the one of Wikipedia as you may desire. The PR-AUC can be thought of as the probability that a classifier will rank a randomly chosen "positive" instance (from the retrieved predictions) higher than a randomly chosen "positive" one (from the original positive class). It should be noted that this is based on my own interpretation and may be subject to error.

In another Wikipedia page, the following text is relevant "precision (also called positive predictive value) is the fraction of retrieved instances that are relevant, while recall (also known as sensitivity) is the fraction of relevant instances that are retrieved".

PR-Curve is a very important metric, especially, when dealing with imbalance datasets. I would advise looking at this study for further details.

From a different persepctive, I would say that when both sensitivity and precision are of importance to the experimenter, then we can think of them as exploration and exploitation terms, respectively. Basically, you may limit strictly the threshold of predictions over the positive class, allowing a very high precision but at the cost of less exploration (i.e. lower sensitivity) (or new insights about the the positive class). Relaxing this constraint, can allow for exploring those cases that we predict as positives and never thought they are.

One would want to always maximize both but for certain cases, this might be very difficult and in some applications, sacrificing one towards enhancing the other is preferable.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ I'm not sure what you say in the first paragraph works out. What does "randomly chosen "positive" instance (from the retrieved predictions)" mean when there are potentially different thresholds (i.e. just using threshold of 0.5 certainly does not reproduce the AuPRC)? $\endgroup$
    – Björn
    Commented Jul 21, 2020 at 13:46

Not the answer you're looking for? Browse other questions tagged or ask your own question.