3
$\begingroup$

I am using the pROC package in R to generate ROC curves. Using the "coords" function, I can extract the sensitivity (Se) , specificity (Sp), negative predicted value (NPV) and positive predicted value (PPV) for different thresholds. I also calculated the Se, Sp, NPV and PPV for some thresholds using the Caret package to compare.

I am a bit confused as the Se an Sp given by the pROC package are actually the NPV and PPV given by the Caret package, respectively (and conversely, the NPV and PPV given by pROC are the Se and Sp in Caret). Any explanations?

Improve this question
$\endgroup$
1
  • $\begingroup$ We have the pROC author on here, and it would be nice to get an answer from such a source! // For reasons I cannot recall, I have begun to prefer the PRROC package to pROC after using pROC for years. (I think it’s because pROC won’t give $AUC<0.5$, even for awful predictions. I get the rationale, but I’d still like to know what’s happening and not have the software make decisions for me.) $\endgroup$
    – Dave
    Mar 15 at 12:09

1 Answer 1

Reset to default
3
$\begingroup$

As @Dave indicated in the comments above, pROC attempts to detect if the positive group displays higher or lower values of the predictor than the control group. This can be controlled with the direction argument.

Caret doesn't do such a detection, and will assume that negative observations have lower values.

It turns out, when you reverse the direction of comparisons, you replace SE and SP with NPV and PPV, respectively.

An other potentially confounding factor is: which group is the negative, and which one is the positive? The two packages have a slightly different logic to figure this out, which can lead to similar discrepancies. In pROC this can be controlled with the levels argument, and in caret with positive

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.