2
$\begingroup$

I am plotting the ROC curve for my classification problem. The results I am getting for the problem are TPR ranges from 0 to 1, but the FPR ranges from 0 to 0.02. I have plotted the ROC curve by taking the same range and it looks like the normal ROC curve (See Figure 1). The issue with the curve is when I calculate the AUC, it seems very low (i.e. 0.33).

As the goal of my classification problem is to minimize the False-Positives, is it possible to plot the ROC curve by changing the X-axis (FPR) range? If possible then how do I interpret the plot?. If not possible How do I go ahead with the same results?

Figure 1. ROC curve

Improve this question
$\endgroup$
6
  • $\begingroup$ Did you plot the blue line yourself? How did you calculate the AUC? And from the plot, it looks like 0.02, not 0.2. $\endgroup$
    – gunes
    Commented Jun 10, 2022 at 5:09
  • $\begingroup$ Why don't you have the full range from zero to one of FPR for your classification problem? And if you would really only want to minimize the FPR, then you could just never classify anything as positive. $\endgroup$
    – frank
    Commented Jun 10, 2022 at 6:15
  • $\begingroup$ @gunes Yes, the value is 0.02, not 0.2. Thank you for pointing out the mistakes. The blue line is for random probabilities. AUC is calculated from FPR and TPR values using AUC function in sklearn. $\endgroup$
    – Rajesh Ahir
    Commented Jun 10, 2022 at 6:57
  • $\begingroup$ @frank For any classification problem the main goal is to classify the sample s in such as way that the amount of FP and FN can be minimized. In my case, the FPR ranges from 0 to 0.02. As for theft detection problems, the large number of FPs may create a problem for the power utilities as on-site inspection is required once the customer is identified as malicious. For the same case, TPs should be as large as possible. $\endgroup$
    – Rajesh Ahir
    Commented Jun 10, 2022 at 7:02
  • $\begingroup$ This is most likely a simple coding mistake, it should always be possible to have a 100% FP rate (by having the model always predict "positive"). Could you post the relevant code? $\endgroup$
    – Eoin
    Commented Jun 10, 2022 at 8:17

1 Answer 1

Reset to default
1
$\begingroup$

IT MEANS YOU ARE CRUSHIN’ IT, BUT WATCH OUT FOR OVERFITTING

If you are able to achieve a high true positive rate without having a high false positive rate, this means that your model is quite good at distinguishing between the two categories.

In your case, you seem to get a perfect true positive rate when the false positive rate is $2\%$. This sounds to me like a case where you achieve a high true positive rate while also having a low false positive rate.

If you plot the ROC curve across all false positive rates like a standard ROC curve plot does, you will see a very steep upward slope to the left that levels out at a perfect true positive rate for almost the entire plot. This corresponds with the area under the curve being close to one, and one indicates perfect separation between the classes.

There are always overfitting concerns when you do the kind of modeling that you are doing, and performance that seems “too good to be true” should lead you to be skeptical. However, the goal is to be able to separate your two categories. If you are able to get the nearly perfect separation indicated by the area under the ROC curve being close to one, this indicates good performance!

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.