0
$\begingroup$

When dealing with a binary classification problem, where the decision function threshold is being varied from 0 to 1 at step 0.1:

  1. When calculating the Area Under the Curve (AUC) for a ROC Curve plot (x: FPR, y: TPR) is the result AUC value equal for both classes?

  2. When calculating the Area Under the Curve (AUC) for a PR (Precision-Recall) Curve plot (x: TPR, y: PPV) is the result AUC value equal for both classes?

I am not asking about any specific framework or method of calculation, just whether to expect the same AUC values for each class or not when using ROC AUC or PR AUC?

|cite|improve this question
$\endgroup$
1
$\begingroup$

You have several misconceptions about ROC and PR curves.

First, ROC and PR curves indicate how well a binary classifier separate positive and negative examples. Hence, you have a single AUC per classifier, not per class.

Second, the following assertion is incorrect:

where the decision function threshold is being varied from 0 to 1 at step 0.1

The ROC and PR curves show how the classifier performs on all thresholds. Depending on the output of your classifier that could mean between $-\infty$ to $+\infty$. There is no stepping, though in practice you will only test the thresholds you observed, effectively making steps.

I suggest you check out the following question and answer for a more thorough overview of ROC curves : Understanding ROC curve.

|cite|improve this answer
$\endgroup$
  • 1
    $\begingroup$ Why can't you have a different AUROC and AUPRC for each class? Weka, for example, has functions that compute a threshold curve separately for each class. Depending on which class is of interest, you can have a different TPR, FPR, Precision, and Recall at each point. Now, it may be possible the the areas under these curves will be the same in either case for a binary classification problem (Still trying to figure that out). $\endgroup$ – rbx Jan 19 '18 at 17:34
  • 1
    $\begingroup$ @rbx no you can't. A binary classifier has a single threshold to discriminate between the classes. $\endgroup$ – Calimo Jan 19 '18 at 18:07
  • 1
    $\begingroup$ I understand that there is a single threshold to discriminate the two classes. I'm talking about how TPR, FPR, Precision, and Recall are computed. Take the confusion matrix with TP = 129, FP = 54, FN = 55, and TN = 92, and compute those 4 measures. Then, flip the confusion matrix, so TP = 92, FP = 55, FN = 54, and TN = 129. You will get different values for those 4 measures. Those are each a point on a different curve. The AUROC should be the same for each curve, but the AUPRC will be different. It's possible I am missing something; please let me know if I am. $\endgroup$ – rbx Jan 19 '18 at 19:57
  • 1
    $\begingroup$ @rbx "flip the confusion matrix" = did you just switch your labels? A binary classifier assumes the labels are known. If you change the truth then you have a new classifier, and different ROC and PR curves. $\endgroup$ – Calimo Jan 19 '18 at 22:02
  • $\begingroup$ @Calimo isn't this a glass half full or half empty problem? Who says which class is positive and which is negative? How does swapping positive with negative produce a new classifier when it is a binary problem? It is just a human readable label without any intrinsic computational or statistical meaning? As far as I can see, PR is for the positive class, but wouldn't that hide terrible poor performance for the negative class, and thus give very misleading analysis based off of the PR curve? Thanks so much. $\endgroup$ – Aalawlx Jan 20 '18 at 2:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.