2
$\begingroup$

If we dont have access to model and have just actual and predicted labels without probabilities, is it still be possible to plot AUC/ROC curve.

For example can we have the curve from the following information (>1000 values in array in actual)

actual = ["C1","C1","C2","C1","C2"]
predicted = ["C2","C1","C2","C1","C1"]

Or is it necessary to have access to probabilities instead of predicted labels?

$\endgroup$
5
  • 1
    $\begingroup$ Your example gives one point on the ROC curve. You can invent two other points, one with all the predictions C1 and another with all the predictions C2. Whether you think three points leading to two line-segments is really an ROC curve is up to you; I would say probably not, as I think you should be adjusting discrimination thresholds with a little more sophistication than this $\endgroup$
    – Henry
    Commented Aug 5, 2020 at 14:39
  • 1
    $\begingroup$ If you assign C1 and C2 distinct numerical values, then you can follow the instructions here stats.stackexchange.com/questions/145566/… to draw a three-point ROC curve with AUC $\approx 0.583$. $\endgroup$
    – Sycorax
    Commented Aug 5, 2020 at 16:32
  • $\begingroup$ @Sycorax, thankyou for the comment, but where will I bring the probability(0-1) that author has as "predicted retention status" in this question, I don't have access to such number $\endgroup$
    – A.B
    Commented Aug 6, 2020 at 3:02
  • $\begingroup$ You don't need it, because ROC curves and AUC are statistics of ranks. Replace all instances of C1 with $0$ and all instances of C2 with $1$. That's what I mean by "assign C1 and C2 distinct numerical values." $\endgroup$
    – Sycorax
    Commented Aug 6, 2020 at 3:10
  • $\begingroup$ Okay thanks, I need to replace in both actual and predicted? $\endgroup$
    – A.B
    Commented Aug 6, 2020 at 3:30

1 Answer 1

2
$\begingroup$

ROC curves are built by varying the cutoff threshold and calculating the sensitivity and specificity for each threshold by checking which labeled points have predicted values above and below the threshold.

If you do not have the probabilities (or some other range of values, like the log-odds from a logistic regression), then you have minimal ability to vary the sensitivity and specificity. Either you classify everything one way to achieve perfect sensitivity with zero specificity, classify everything the other way to achieve zero sensitivity with perfect specificity, or classify according to the predicted labels to give some sensitivity-specificity combination (which might represent quite good performance). This leads to the three-point ROC curve discussed in the comments.

$\endgroup$
1
  • 1
    $\begingroup$ Not horizontal: thresholds below 0 and above 1 give points at the corners of the ROC space, and then the actual (FPR, TPR) of the hard classifier gives one interior point. So it's a three-point "curve" as @Henry comments. $\endgroup$ Commented Aug 5, 2020 at 16:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.