7
$\begingroup$

Wikipedia says that an interpretation of the area under the ROC curve is: "the area under the curve is equal to the probability that a classifier will rank a randomly chosen positive instance higher than a randomly chosen negative one".

But is it the same interpretation of the area under the PR curve? If not, can you please give me an intuitive interpretation for it like the above?

Edit: PR == Precision-Recall

$\endgroup$
  • 3
    $\begingroup$ Because I wasn't sure exactly what you meant by "PR curve" (the world has too many acronyms to be confident of any of them), I googled it and one of the top hits is a presentation answering your question $\endgroup$ – whuber Oct 16 '14 at 23:19
  • $\begingroup$ @whuber see edit. $\endgroup$ – Jack Twain Oct 17 '14 at 7:50
1
$\begingroup$

The area under the PR-Curve is ill-defined. Because there is no well-defined precision at recall 0: you get a division by zero there.

You also cannot close this gap easily - it may be anything from 0 to 1, depending on how well your retrieval works.

There is a common approximation to this - AveP, average precision.

$\endgroup$
  • $\begingroup$ When recall is 0, not necessarily, you get a division by zero. A recall = 0 => True Positives = 0. However, the classifier might predict many cases as positives without hiting a true one. In such case, False Positives is defined and precision will be 0 also. What do you think? $\endgroup$ – soufanom Oct 17 '14 at 20:15
  • $\begingroup$ If you do this, the perfect method will not have a score of 1 either. IMHO, this is not the proper way to resolve this. Just stick to AveP. $\endgroup$ – Anony-Mousse Oct 18 '14 at 11:09
  • $\begingroup$ @Anony-Mousse can you please see my question here: stats.stackexchange.com/questions/123697/… $\endgroup$ – Jack Twain Nov 12 '14 at 17:34
0
$\begingroup$

Well, I will try to give some intuition close to the one of Wikipedia as you may desire. The PR-AUC can be thought of as the probability that a classifier will rank a randomly chosen "positive" instance (from the retrieved predictions) higher than a randomly chosen "positive" one (from the original positive class). It should be noted that this is based on my own interpretation and may be subject to error.

In another Wikipedia page, the following text is relevant "precision (also called positive predictive value) is the fraction of retrieved instances that are relevant, while recall (also known as sensitivity) is the fraction of relevant instances that are retrieved".

PR-Curve is a very important metric, especially, when dealing with imbalance datasets. I would advise looking at this study for further details.

From a different persepctive, I would say that when both sensitivity and precision are of importance to the experimenter, then we can think of them as exploration and exploitation terms, respectively. Basically, you may limit strictly the threshold of predictions over the positive class, allowing a very high precision but at the cost of less exploration (i.e. lower sensitivity) (or new insights about the the positive class). Relaxing this constraint, can allow for exploring those cases that we predict as positives and never thought they are.

One would want to always maximize both but for certain cases, this might be very difficult and in some applications, sacrificing one towards enhancing the other is preferable.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.