As title, I haven't found a post explain about this, i.e. In the following diagram, for a point on a "better" ROC curve and I move that point along the curve toward the top-right corner, why both TPR and FPR approach 1? And what's the meaning of moving along a given curve?
1 Answer
An ROC curve shows a classification model over all possible thresholds, not a single classifier. Any point on the ROC curve represents a particular model instantiation with a particular threshold, and particular outputs. You can think of your classification model as producing a score for every sample, and to get a single classifier, you need to pick a threshold for calling things positive/negative.
The ROC curve shows the model's performance at every possible threshold - you start with a very low threshold at the bottom left, calling everything negative (FPR=0, TPR=0). As you raise the threshold, you move along the curve as you start to identify true positives. In the ideal case with the perfect model, you'll find a threshold that identifies all the positives and none of the negatives, getting you FPR=0, TPR=1. If you increase the threshold too much, the model may get somewhat worse, as you'll still identify all the positives, but also some of the negatives (TPR=1, FPR>0).
When you're all the way in the top right corner, you've set your threshold higher/lower than any score in the dataset, and you simply classify everything as positive. At that point the scores are irrelevant, so it doesn't matter if you have the "perfect" classification model that produces meaningful scores, or a random classifier that produces meaningless scores. If you set your threshold higher/lower than any of the classifier scores, the scores don't actually matter.
As you can see, you don't want a classifier that falls in the top right corner - it's a naive classifier that just calls everything positive, regardless of any input features. You want your classifier to be in the top left corner, at TPR=1 and FPR=0.
-
-
1$\begingroup$ @Niing To build an ROC curve, you need a classification model that outputs a score. You can then set a threshold for the score, and call things above/below it positive/negative to get a binary classifier. Moving the threshold will yield different classification outputs for the same underlying model. Depending on if your positives have a higher score than negatives, or vice versa, you can think of it as starting with a low threshold and raising it, or starting with a high one and lowering it. $\endgroup$ Commented May 11, 2021 at 16:30
-
$\begingroup$ Thank you very much! Your edit helps a lot. $\endgroup$– NiingCommented May 11, 2021 at 16:34
-
$\begingroup$ This graph is indeed very hard for me, but I think I'm getting some idea from your answer. E.g. Say I'm on the perfect curve, even at point (FPR=0, TPR=1) I can still increase the threshold, but then I will only move horizontally. $\endgroup$– NiingCommented May 11, 2021 at 16:38
-
$\begingroup$ @Niing Exactly. With most classifiers, there will be a tradeoff, so the "best" threshold value will depend on your purpose. For the blue curve, for example, you may want to be near the left side to ensure few false positives at the cost of missing some true positives. But for a medical diagnostic, for example, you might want to set the threshold so that you're nearer the top of the plot, to catch almost all instances of disease at the cost of some false positives. With the perfect classifier, there's really only one logical choice of threshold, which puts you in the top left corner. $\endgroup$ Commented May 11, 2021 at 16:41