1
$\begingroup$

I have this confusión matrix results:

True Negatives : 725
False Positives : 74
False Negatives : 62
True Positives : 139

              precision    recall  f1-score   support

       False       0.92      0.91      0.91       799
        True       0.65      0.69      0.67       201

    accuracy                           0.86      1000
   macro avg       0.79      0.80      0.79      1000
weighted avg       0.87      0.86      0.87      1000

And plotting the ROC curve I got this:

enter image description here

AUC ROC=1.000

MCC (Matthews correlation coefficient) = 0.6412

Is clear that according to the literature, the classifier is doing perfect, but I'm skeptical considering that I got the same image (with a different confusion matrix) with a threshold of 0.5 and even with 0.2.

I'm not sure if I'm doing well and if AUC ROC is the more appropriated to measure my classifier. Considering this is my first time dealing with AUC ROC curves, what else should I do to resolve my concerns?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

I think the plot is wrong if you show your code I can help you make it. I say this because you have false positives and false negatives.

Improve this answer
$\endgroup$
4
  • $\begingroup$ I'm using scikit-learn. The code is simple: fpr, tpr, _ = roc_curve(testy, probs) where testy is a list the binary data (0,1) and probs are the probabilities for each element in the list. After that, pyplot.plot(fpr, tpr, linestyle='--'). $\endgroup$
    – Fernando Barraza
    Commented Jan 28, 2021 at 17:21
  • $\begingroup$ Since you do classification instead of probs and testy put scores, labels in that order.This is the functon signature way to call ot sklearn.metrics.roc_curve(y_true, y_score) check the docs too scikit-learn.org/stable/modules/generated/… if you are not sure pur the conctent of testy and probs here $\endgroup$
    – partizanos
    Commented Jan 29, 2021 at 21:41
  • $\begingroup$ I found my mistake ...you gave me light on what could be happening. The first parameter in roc_curve is the actual values, I was confusing this with the predicted values. thanks a lot. $\endgroup$
    – Fernando Barraza
    Commented Jan 30, 2021 at 22:55
  • $\begingroup$ Happy it helped please mark my answer as maybe other people with similar ROC issues will find it useful. $\endgroup$
    – partizanos
    Commented Jan 30, 2021 at 23:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.