2
$\begingroup$

I have created a ROC curve for two classifiers on a two-class classification task:

enter image description here

The green curve is from a nearest neighbours classifiers. The red one from a tree boosting.

As we can see both curve cross each other. What can we say in this case? Which one is better? If a curve is completely above another, it is clear that it is better, but I don't know the case for crossings...

Improve this question
$\endgroup$
4
  • $\begingroup$ If you compute the area under the curve the blue seems bigger! See this stats.stackexchange.com/q/103884/77852 and the comments for more information on formal tests $\endgroup$
    – Robert
    Commented Jul 30, 2016 at 1:08
  • $\begingroup$ note that ROC and AUROC are random variables; stats.stackexchange.com/questions/165033/… $\endgroup$
    – user83346
    Commented Jul 30, 2016 at 7:28
  • $\begingroup$ Side note, it would be easier for a reviewer to compare the superficial difference between the curves if the figure was symmetrically scaled. The blue curve does appear to have a greater AUC, but the scaling is not the same on both axes! $\endgroup$
    – hlsmith
    Commented Aug 31, 2016 at 15:19
  • $\begingroup$ @hlsmith The aspect ratio is irrelevant, because a patch of any given size in this plot continues to correspond to the same area even when it is translated. That shows that any visually apparent difference in areas in the plot truly is a difference in the actual areas. $\endgroup$
    – whuber ♦
    Commented Aug 31, 2016 at 16:22

1 Answer 1

Reset to default
4
$\begingroup$

Those curves mean there's a specific rank where both the $\text{TPR}$ and the $\text{FPR}$ values are the same among curves. It's also clear the blue one has higher sensitivity at low specificity, while the red one has higher specificity at low sensitivity. Basically, it would mean one curve is better at predicting positives than the other, but there's of course a tradeoff.

Now, which one is better? Depends on what you define as "better", and also on the intended application. Without further details about the data it's hard to tell, but if you consider the $\text{AUC}$ as a good metric for your problem, then blue seems to be better (not sure, just eyeballing an area). Also, at the $\text{FPR} = \text{TPR}$ line, blue is also better.

Improve this answer
$\endgroup$
3
  • $\begingroup$ note that ROC and AUROC are random variables, in other words the curves depend on the sample; stats.stackexchange.com/questions/165033/… $\endgroup$
    – user83346
    Commented Jul 30, 2016 at 7:28
  • $\begingroup$ Thank you for your answer. Which curve is better at predicting positives? $\endgroup$
    – machinery
    Commented Jul 30, 2016 at 8:23
  • $\begingroup$ @machinery the blue one, it has higher $\text{TPR}$ at $\text{FPR} \geq 0.15$. But you always have to keep in mind the tradeoff, at $\text{FPR} < 0.15$ red has higher $\text{TPR}$, which is, of course, lower than in the first case. $\endgroup$
    – Firebug
    Commented Jul 30, 2016 at 14:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.