3
$\begingroup$

Say I have a classifier that assigns a score to an image based on whether it has a cat in it. The higher the score, the more likely there's a cat in it. But for this classifier, the value of the score is unbounded, and could be any positive number, in principle. Is there a well-defined way to create a ROC curve for this classifier, if all the images have yes/no labels? Just as a traditional ROC curve involves all thresholds between 0 and 1, could a modified ROC curve involve all decision thresholds between 0 and the highest score?

I could normalize the scores to [0,1] by dividing them by the highest score in the set, but I want to avoid that if possible.

$\endgroup$
4
  • 1
    $\begingroup$ Typically a model like this would use some kind of function to squeeze the values into $[0,1]$ e.g. logistic regression. However, I see no reason why you couldn’t slide your threshold up and down the real like and calculate the sensitivity and specificity to plot. $\endgroup$
    – Dave
    Sep 30 '20 at 15:08
  • 1
    $\begingroup$ ROC curves only care about the ranking/the ordering of your images. It doesn't matter how you normalize your scores, the ROC won't change. $\endgroup$ Sep 30 '20 at 15:25
  • 1
    $\begingroup$ "traditional ROC curve involves all thresholds between 0 and 1" Where did you get this information from? It is plain wrong! $\endgroup$
    – Calimo
    Sep 30 '20 at 17:06
  • 1
    $\begingroup$ @LaksanNathan care to write it as an answer? $\endgroup$
    – Calimo
    Sep 30 '20 at 17:07
1
$\begingroup$

The ROC curve plots the False Positive Rate (on the horizontal axis) against the True Positive Rate (on the vertical axis) as your classification threshold varies.

Yes, the two rates will always be between $0$ and $1$. But there is no reason whatsoever that the threshold needs to be in some specified interval, like $[0,1]$, or any other fixed $[a,b]$.

Just start with your threshold at the lowest score you have, at which you will have neither FPs nor TPs, so this is the point at $(0,0)$ of the ROC curve. Then increase the threshold. The FPRs and TPRs will change; plot them. When you end up at the highest score you have, you will have the FPR and the TPR both equal to $1$ and plot the $(1,1)$ point of the ROC curve.

$\endgroup$
1
  • $\begingroup$ Thanks Stephan! $\endgroup$ Oct 10 '20 at 16:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.