3
$\begingroup$

I have a plot of ROC curves for about 5 models. The curves are overlapping, as shown in the attached figure.

enter image description here

Is there a way to still call out the differences between these models in a research paper using a ROC curve, or do I present the AUC values in a metrics table?

Note: when you break the axis and for example use log scale, it is still overlapping because the data values are of the form(Where M1 = model1 and M2 is model 2.):

+--------+---------+------------+---------+
| FPR_M1 | TPR_M1  | FPR_M2     | TPR_M2  |
+--------+---------+------------+---------+
| 0      | 0       | 0          | 0       |
| 0      | 0.99452 | 0          | 0.93296 |
| 0      | 0.99563 | 0          | 0.97548 |
| 0      | 0.99728 | 0          | 0.98833 |
| 0      | 0.99863 | 0          | 0.99995 |
| 0      | 1       | 0          | 1       |
| 1      | 1       | 3.70233E-5 | 1       |
|        |         | 6.17055E-5 | 1       |
|        |         | 8.63878E-5 | 1       |
|        |         | 1.60434E-4 | 1       |
|        |         | 2.34481E-4 | 1       |
|        |         | 3.3321E-4  | 1       |
|        |         | 4.07257E-4 | 1       |
|        |         | 5.18327E-4 | 1       |
|        |         | 7.15784E-4 | 1       |
|        |         | 8.63878E-4 | 1       |
|        |         | 0.00127    | 1       |
|        |         | 0.00202    | 1       |
|        |         | 0.00327    | 1       |
|        |         | 0.00585    | 1       |
|        |         | 0.01319    | 1       |
|        |         | 0.05294    | 1       |
|        |         | 1          | 1       |
+--------+---------+------------+---------+


$\endgroup$

1 Answer 1

5
$\begingroup$

I do not think there is a reason to show these AUC-ROC curves. With AUC scores approximating $1$ all curves are going to look the same and convey the same information. Having a small one-row table will be more than enough (probably in the Appendix even).

I would suggest using another metric/visualisation to communicate meaningful/any differences between model performance characteristics (if relevant).

(And to point at the elephant in the room: AUC-ROC scores so close to $1$ will raise strong suspicions about overfitting the test set. I hope that this is properly addressed in the paper.)

$\endgroup$
1
  • $\begingroup$ Thanks for the suggestion above and for pointing to the elephant in the room. I will consider using another metric and go back to the training pipeline to see if data leakage might lead to overfitting. $\endgroup$
    – cidi30bg
    Mar 5, 2023 at 19:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.