2
$\begingroup$

ROC curve usually looks like the following figure:

ROC curve

If we have enough data, could we safely assume that ROC curve for a model will always be symmetric around the line y = 100 - x? If not, is there any way to make the ROC curve shift up (the red line)?

I am asking this question because my final goal is to optimize sensitivity at a minimum specificity. However, using sensitivity as my evaluation metric seems to be very similar to using AUC. So, I am wondering whether using AUC is highly correlated with using sensitivity as an evaluation metric because you can not intentionally tweak the shape of ROC curve.

I also tried using sample weight, but it seems to only impact the cutoff of the classification model. Please check my attempts with SVM by changing the weight of the positive class.

$\endgroup$
  • 3
    $\begingroup$ It's ROC curve, not AUC curve. AUC is a single statistic you compute from the curve. $\endgroup$ – SmallChess Feb 28 '17 at 6:04
  • 2
    $\begingroup$ No, it won't always be symmetric. It depends on the distribution of the "score" that generates the curve. $\endgroup$ – GoF_Logistic Mar 8 '17 at 21:58
3
$\begingroup$

No, the ROC curve does not need to be symmetric.

Take as an example a dataset with a binary dependent variable $y$ and a single binary independent variable $x$. Assume that $y=1$ whenever $x=1$ and that $y$ is randomly selected when $x=0$, taking value 1 with probability 0.5 and taking value 0 with probability 0.5. Assume $x=1$ for 1/3 of the observations and $x=0$ for 2/3 of the observations.

No matter how much data you have, any sensible model will have an ROC curve like the following:

enter image description here

Basically you can identify half of the positives with no errors using the observations with $x=1$, and the rest of the selection is more or less random. The ROC curve from your model will not be symmetric regardless of the size of the training set.

$\endgroup$
  • $\begingroup$ Do you mean "half of the samples"? "Half of the positives" mean you have 50% accuracy in your positive samples (you have 100%). $\endgroup$ – SmallChess Mar 9 '17 at 1:15
  • $\begingroup$ @StudentT I mean "half of the positives," not "half of the samples." 1/3 of the samples have (x, y)=(1, 1), 1/3 have (x, y)=(0, 1) and 1/3 have (x, y)=(0, 0). You can therefore identify half of the positive samples by labeling everything with x=1 as being a positive observation. This corresponds to the point with TPR=0.5 and FPR=0. $\endgroup$ – josliber Mar 9 '17 at 2:03
  • $\begingroup$ You are right, but it's there any way in model building that we can purposely change the ROC curve? i.e., fixed FPR, optimizing TPR. $\endgroup$ – user2149631 Mar 10 '17 at 0:29
  • $\begingroup$ @user2149631 I'm afraid I don't understand your question. Anyhow, in the example dataset I provided it is pretty clear that no model can improve over the plotted curve, aka that curve represents the maximum achievable TPR for every FPR. $\endgroup$ – josliber Mar 10 '17 at 2:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.