1
$\begingroup$

In their article of 2006, The Relationship Between Precision-Recall and ROC Curves, J. Davis & M. Goadrich show that ROC (TPR vs FPR) and Precision-Recall (PR) curves have one-to-one correspondence. In particular, the authors prove equivalence of a classifier dominating another in either space (for a given mix of classes, A dominates B in ROC space <=> A dominates B in PR space). Now, the ROC curves are not affected by changing the mix of positive and negative classes, yet the PR curves are. Therefore, the equivalence is really not there -- because ROC dominance (of A over B) implies PR dominance for any mix (since the ROC curve remains the same) whereas PR dominance implies ROC dominance for a specific mix only. However, for a different mix it's conceivable that B may dominate A in the PR space. Is there some kind of a paradox here?

$\endgroup$
  • $\begingroup$ @user777, yes indeed, but that's exactly at the core of the problem (or confusion?) and it's mentioned explicitly above ("for a given mix of classes"). $\endgroup$ – dnqxt Jan 28 '16 at 23:12
  • 1
    $\begingroup$ Sorry, I had to re-read this a few times to understand what you mean. Dominance like an inequality, so if A >B in ROC space, then we can use the conclusion that A > B in PR space. Since PR space is sensitive to class composition, the equivalence result is maintained in the presence of class skew iff AUPR is lower for class-skewed data. The result of equivalence between the two is still maintained, though: A>B. $\endgroup$ – Sycorax Jan 29 '16 at 3:50
  • $\begingroup$ @user777 Thank you for your comment and time. For completeness/clarity it's helpful to refer to the original article google.com/… $\endgroup$ – dnqxt Jan 29 '16 at 8:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.