0
$\begingroup$

With respect to the pROC package for R(https://rdrr.io/cran/pROC/man/ggroc.html).

library(pROC)
# Create a basic roc object
data(aSAH)
rocobj <- roc(aSAH$outcome, aSAH$s100b)
roc_obj$thresholds
 [1]  -Inf  1.50  2.50  3.50  4.50  5.50  6.50
 [8]  7.50  8.50  9.50 10.75 12.25 13.50 14.50
[15] 15.50 16.50 17.50 18.50 19.50   Inf
roc_obj$sensitivities
  [1] 0.64919355 0.64919355 0.64919355 0.64919355
  [5] 0.64919355 0.64919355 0.64919355 0.64658635
  [9] 0.64658635 0.64658635 0.64658635 0.64658635
 [13] 0.64658635 0.64658635 0.64658635 0.64658635
 [17] 0.64400000 0.64400000 0.64400000 0.64400000
 [21] 0.64143426 0.64143426 0.63888889 0.63888889
 [25] 0.63888889 0.63492063 0.63492063 0.63492063
 [29] 0.63492063 0.63095238 0.62845850 0.62450593
 [33] 0.62055336 0.61568627 0.61176471 0.60784314
 [37] 0.60784314 0.60000000 0.60000000 0.59765625
 [41] 0.59375000 0.58593750 0.58203125 0.57421875
 [45] 0.57421875 0.56640625 0.56201550 0.55038760
 [49] 0.53488372 0.51162791 0.50000000 0.49615385
 [53] 0.48659004 0.47892720 0.46360153 0.45593870
 [57] 0.45210728 0.44274809 0.42585551 0.41825095
 [61] 0.41064639 0.39923954 0.39015152 0.38257576
 [65] 0.37735849 0.36981132 0.36226415 0.35471698
 [69] 0.34339623 0.34339623 0.33962264 0.32830189
 [73] 0.31698113 0.29811321 0.29433962 0.28679245
 [77] 0.27924528 0.27067669 0.25563910 0.24436090
 [81] 0.23684211 0.23684211 0.22932331 0.22556391
 [85] 0.21428571 0.19172932 0.18421053 0.17669173
 [89] 0.16541353 0.13909774 0.13909774 0.12781955
 [93] 0.12030075 0.10526316 0.07518797 0.06390977
 [97] 0.06015038 0.05263158 0.04135338 0.01879699
[101] 0.00000000
> roc_obj$specificities
  [1] 0.9179612 0.9205811 0.9237084 0.9253372
  [5] 0.9287434 0.9302885 0.9337176 0.9347722
  [9] 0.9362722 0.9377395 0.9388730 0.9408679
 [13] 0.9424358 0.9448931 0.9469697 0.9498818
 [17] 0.9522910 0.9532357 0.9552098 0.9562353
 [21] 0.9581570 0.9586854 0.9615565 0.9620431
 [25] 0.9639513 0.9654206 0.9664492 0.9669306
 [29] 0.9674570 0.9679517 0.9679517 0.9684747
 [33] 0.9694019 0.9694019 0.9699074 0.9713228
 [37] 0.9727357 0.9736842 0.9737206 0.9737569
 [41] 0.9755985 0.9770009 0.9775126 0.9780119
 [45] 0.9794050 0.9812357 0.9812443 0.9821674
 [49] 0.9826325 0.9826484 0.9831358 0.9840692
 [53] 0.9849932 0.9854678 0.9859347 0.9864069
 [57] 0.9873360 0.9886980 0.9896163 0.9896256
 [61] 0.9900812 0.9900856 0.9905405 0.9905405
 [65] 0.9909950 0.9914491 0.9918992 0.9932524
 [69] 0.9937022 0.9937079 0.9937135 0.9946164
 [73] 0.9955137 0.9964109 0.9968596 0.9968610
 [77] 0.9973118 0.9973154 0.9973166 0.9977639
 [81] 0.9977639 0.9982119 0.9982127 0.9982127
 [85] 0.9982127 0.9982143 0.9986613 0.9986613
 [89] 0.9986613 0.9991079 0.9991079 0.9991083
 [93] 0.9991083 0.9991087 0.9991095 0.9995550
 [97] 1.0000000 1.0000000 1.0000000 1.0000000
[101] 1.0000000

Each point on the ROC curve represents a sensitivity/specificity pair corresponding to a particular decision threshold. That means if the output probability value is more than the threshold than the class associated with it become the called class. For example, if the threshold is set to 0.5 and the output probability of a case being diabetic is 0.75 then the case is considered as a disease.

But when you look at the thresholds in the above examples then it ranges from -inf to +inf including values greater than 1. How can thresholds be greater than 1 when the probability values range from 0 to 1?

Also, I am not able to connect the meaning of -inf and +inf, will it behave the same as max and min-cut numbers of probability like 0 and 1.

$\endgroup$
6
$\begingroup$

But when you look at the thresholds in the above examples then it ranges from -inf to +inf including values greater than 1. How can thresholds be greater than 1 when the probability values range from 0 to 1?

This appears to stem from a typo in your code.

rocobj <- roc(aSAH$outcome, aSAH$s100b)
roc_obj$thresholds

Note the underscore in roc_obj that is not present in rocobj.

Errors like this are why it's best practice to clear your workspace or start a new session before attempting to construct an example.

But when you look at the thresholds in the above examples then it ranges from -inf to +inf including values greater than 1. How can thresholds be greater than 1 when the probability values range from 0 to 1?

First, the data you're using do not vary between (0,1), so it's not a probability.

summary(aSAH$s100b)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  0.030   0.090   0.140   0.247   0.330   2.070 

This is irrelevant, though, because ROC curves don't assume that inputs are probabilities. ROC curves just care about ranking the inputs; as long as the inputs can be ordered, it's fine.

If you print rocobj$thresholds instead of roc_obj$thresholds, you'll see that it has values in the range of the data except for the endpoints -Inf and Inf. The thresholds at the endpoints are -Inf and Inf because ROC curves are defined as monotonic increasing curves from (0,0) to (1,1); you need to have thresholds outside of the range of your data to achieve (0,0) and (1,1); hence -Inf and Inf are used. The interpretation of operating points at (0,0) and (1,1) is no different, because these correspond to the TPR and FPR of naïve models which either never raise the alarm or always raise the alarm.

rocobj$thresholds
 [1]  -Inf 0.035 0.045 0.055 0.065 0.075 0.085 0.095 0.105 0.115 0.125 0.135
[13] 0.145 0.155 0.165 0.175 0.185 0.205 0.225 0.235 0.245 0.255 0.265 0.275
[25] 0.290 0.310 0.325 0.335 0.345 0.365 0.395 0.420 0.435 0.445 0.455 0.465
[37] 0.475 0.485 0.495 0.510 0.540 0.570 0.640 0.705 0.725 0.755 0.795 0.840
[49] 0.910 1.515   Inf
$\endgroup$
  • $\begingroup$ thanking for pointing out my mistake. Can you please elaborate more about the infinity thing. So is it like a manual addition to the thresholds, despite whatever values you take. Which is basically for sensitivity and specificity (1,0) and (0,1) at respective infinite thresholds. $\endgroup$ – Dhwani Dholakia Sep 24 at 18:41
  • $\begingroup$ Imagine that you have a model that produces finite outputs; your model produces outputs in $(0,1)$, which is a special case of producing finite outputs. You have a threshold of $-\infty$. What's the TPR and FPR at this threshold? Will your model ever make a prediction that's less than $-\infty$? What's the TPR and FPR for a model with a threshold at $\infty$? $\endgroup$ – Reinstate Monica Sep 24 at 18:49
  • $\begingroup$ @Sycorax Does it mean when there is -inf the sensitivity will always be 1 and +Inf the sensitivity will be 0 and similarly vice versa for specificity. So in cases where you are manually finding sensitivity and specificity based on various thresholds, you should consider + inf and -inf threshold values also? $\endgroup$ – Dhwani Dholakia Sep 24 at 18:59
  • 1
    $\begingroup$ If you never detect, you have a TPR of 0 and an FPR of zero by definition. Likewise, if you always detect, you'll always have a TPR of 1 and an FPR of 1. $\endgroup$ – Reinstate Monica Sep 24 at 19:04
  • $\begingroup$ thanks for the clarification. $\endgroup$ – Dhwani Dholakia Sep 24 at 20:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.