I produced the ROC curve for a continuous biomarker used as a classifier toward a binary class (absence/presence of a disease according to another biomarker), using the ROCR package in R.

I have a basic knowledge of the statistics of classification functions and a good knowledge of R, but I did not expect the curve in the picture. How to interpret this behaviour? First, the curve plateaus earlier than, let's say the honest thing, the ROC curves you see in books. Second, it actually goes into the area in which a classifier performs worse than a random classifier (that gives the same classification X% of the time). Does it just mean that the classifier's performance is poor (in particular, its sensitivity is poor as it gives several false negatives at all thresholds) and that below a certain threshold you should reverse its classification order for it to make sense?

One possibly relevant fact: in the dataset I used, because of the way it was collected retrospectively, there are no cases of disease absence with a low level of this biomarker. We only have, roughly speaking, low biomarker/disease present; high biomarker/disease present; low biomarker/disease present. Does it explain anything?

enter image description here

  • 1
    $\begingroup$ I've had ROCs look like this before when I had an error in the code (my labels were mixed up or something along those lines). Check for any bugs you may have. $\endgroup$ – bdeonovic Jan 19 '16 at 20:08
  • $\begingroup$ I will check everything, that's indeed the most likely explanation. $\endgroup$ – torwart Jan 19 '16 at 20:16
  • 1
    $\begingroup$ It looks like TPR has an asymptote at 1/2, so I'd check TPR computations first. $\endgroup$ – Sycorax Jan 20 '16 at 0:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.