1
$\begingroup$

I am trying to do a binary classification (0 and 1), and in this case, the end goal is to maximise the True Positives (i.e. maximum no. of 1s should be predicted correctly, even if it gives rise to significant False Positives). I am obtaining a ROC curve, with Area Under Curve (AUC) for Class 0 being 0.55 and Class 1 being 0.75.

From what I have seen over SE and over the Internet, a value of AUC of 0.75 is a fair value, however, many resources mention that a value in the range 0.50-0.60 etc. might indicate a worthless/random classifier, but I definitely want to maximise the correct predictions for Class 1 instead of losing out on predictions for 1s which may actually belong to 1s. I would like to know that in this case, is the AUC I obtain of any significance, or does it show that the classifier is not useful. Any advice in this regard is highly appreciated.

EDIT The ROC Curve for the problem which I developed in Python is given below.enter image description here

$\endgroup$

closed as unclear what you're asking by Stephan Kolassa, user158565, mdewey, Sycorax, Siong Thye Goh May 4 at 2:52

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 4
    $\begingroup$ The trivial way to maximize TPs is to declare everything a positive. Because this observation is completely trivial, I think it's more likely that you really want to find a particular trade-off between TP and FP. Can you shed some light on what your criteria are for a good model, and how this model falls short? Also, what's AUC per class? ROC AUC describes the relative ranks of 2 classes in a single number. $\endgroup$ – Sycorax May 2 at 21:24
  • 2
    $\begingroup$ You probably have an elbow because you used predict instead of predict_proba (or similar). But reporting a different ROC AUC for each class doesn't make sense. You'll have to explain what you're doing in more detail for this to be answerable. $\endgroup$ – Sycorax May 3 at 13:22
  • 1
    $\begingroup$ I haven't used skplt. Maybe the documentation will explain why it's doing this. The plot suggests that all of the predictions are identical for each of the classes. Is that true? $\endgroup$ – Sycorax May 3 at 15:46
  • 1
    $\begingroup$ I can't say anything about what the plot shows because it has no relationship to any ROC procedure that I've seen before. My comment from yesterday about trivially maximizing true positives remains just as valid now as it was then because you've made no elaboration about what limits, if any, you have for false positives. $\endgroup$ – Sycorax May 3 at 15:58
  • 1
    $\begingroup$ Whether it's good or bad depends on the context. If your test is incorrect, and you say a patient is ill and needs to have his foot amputated, that's a terrible outcome. If your test is incorrect, and the patient takes an unnecessary dosage of vitamin C, the consequences are much less severe. $\endgroup$ – Sycorax May 3 at 16:10