3
$\begingroup$

ROC curve usually looks like the following figure:

ROC curve

If we have enough data, could we safely assume that ROC curve for a model will always be symmetric around the line y = 100 - x? If not, is there any way to make the ROC curve shift up (the red line)?

I am asking this question because my final goal is to optimize sensitivity at a minimum specificity. However, using sensitivity as my evaluation metric seems to be very similar to using AUC. So, I am wondering whether using AUC is highly correlated with using sensitivity as an evaluation metric because you can not intentionally tweak the shape of ROC curve.

I also tried using sample weight, but it seems to only impact the cutoff of the classification model. Please check my attempts with SVM by changing the weight of the positive class.

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ It's ROC curve, not AUC curve. AUC is a single statistic you compute from the curve. $\endgroup$
    – SmallChess
    Commented Feb 28, 2017 at 6:04
  • 2
    $\begingroup$ No, it won't always be symmetric. It depends on the distribution of the "score" that generates the curve. $\endgroup$
    – GoF_Logistic
    Commented Mar 8, 2017 at 21:58

1 Answer 1

Reset to default
4
$\begingroup$

No, the ROC curve does not need to be symmetric.

Take as an example a dataset with a binary dependent variable $y$ and a single binary independent variable $x$. Assume that $y=1$ whenever $x=1$ and that $y$ is randomly selected when $x=0$, taking value 1 with probability 0.5 and taking value 0 with probability 0.5. Assume $x=1$ for 1/3 of the observations and $x=0$ for 2/3 of the observations.

No matter how much data you have, any sensible model will have an ROC curve like the following:

enter image description here

Basically you can identify half of the positives with no errors using the observations with $x=1$, and the rest of the selection is more or less random. The ROC curve from your model will not be symmetric regardless of the size of the training set.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Do you mean "half of the samples"? "Half of the positives" mean you have 50% accuracy in your positive samples (you have 100%). $\endgroup$
    – SmallChess
    Commented Mar 9, 2017 at 1:15
  • $\begingroup$ @StudentT I mean "half of the positives," not "half of the samples." 1/3 of the samples have (x, y)=(1, 1), 1/3 have (x, y)=(0, 1) and 1/3 have (x, y)=(0, 0). You can therefore identify half of the positive samples by labeling everything with x=1 as being a positive observation. This corresponds to the point with TPR=0.5 and FPR=0. $\endgroup$
    – josliber
    Commented Mar 9, 2017 at 2:03
  • $\begingroup$ You are right, but it's there any way in model building that we can purposely change the ROC curve? i.e., fixed FPR, optimizing TPR. $\endgroup$
    – user2149631
    Commented Mar 10, 2017 at 0:29
  • $\begingroup$ @user2149631 I'm afraid I don't understand your question. Anyhow, in the example dataset I provided it is pretty clear that no model can improve over the plotted curve, aka that curve represents the maximum achievable TPR for every FPR. $\endgroup$
    – josliber
    Commented Mar 10, 2017 at 2:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.