Diagonal in ROC plot?

I've used the R ROCR package to generate an ROC plot. I can see a diagonal in my plot. However, the ROC curve that I see other people generated has only horizontal and vertical lines.

Why would I get a plot like that? Is that possible to have a diagonal line in a legal ROC plot?

ROC curve I see in the literature looks like this:

My generated ROC plot is:

-
What problem are you trying to solve that made ROC curves have anything to do with it? – Frank Harrell Jan 16 at 18:52

Assume that you have the following result:

score label
1.000 positive
0.900 negative
0.900 positive
0.900 negative
0.500 negative
0.200 positive


Manually plot the ROC curve for the possible thresholds of 1.0, 0.9, 0.5, 0.2 and you do get a sloped part.

The reason are duplicate scores.

Beware, there are some poor implementations of ROC out there. I've seem some that only sample values (usually you can recognize this because they have very evenly spaced steps), and I've seen implementations that simply sort the data and then take the nth object - ignoring duplicate scores. If the input data is presorted by label, this causes results to be much better than expected. This can be detected by using a data set where all scores are 0 - the only correct result then is the diagonal line and a AuC of 0.5

-
@NickCox sorry crappy touchscreens. presorted. If the data was sorted by label at some point, and scores are duplicate, then often all the positives or all the negatives come first. – Anony-Mousse Jan 16 at 20:56

Yes, this is "legal". If the jump from one threshold to the next raises the amount of false positives and false negatives together the result is a diagonal line. Two reasons that might happen:

1. You have 2 observations with same threshold but with different ground truth
2. The resolution between 2 thresholds is large enough - in that case you may also check a threshold between the two.
-