How to decide threshold values for AUC?

Question

With respect to the pROC package for R(https://rdrr.io/cran/pROC/man/ggroc.html).

library(pROC)
# Create a basic roc object
data(aSAH)
rocobj <- roc(aSAH$outcome, aSAH$s100b)
roc_obj$thresholds
 [1]  -Inf  1.50  2.50  3.50  4.50  5.50  6.50
 [8]  7.50  8.50  9.50 10.75 12.25 13.50 14.50
[15] 15.50 16.50 17.50 18.50 19.50   Inf

roc_obj$sensitivities
  [1] 0.64919355 0.64919355 0.64919355 0.64919355
  [5] 0.64919355 0.64919355 0.64919355 0.64658635
  [9] 0.64658635 0.64658635 0.64658635 0.64658635
 [13] 0.64658635 0.64658635 0.64658635 0.64658635
 [17] 0.64400000 0.64400000 0.64400000 0.64400000
 [21] 0.64143426 0.64143426 0.63888889 0.63888889
 [25] 0.63888889 0.63492063 0.63492063 0.63492063
 [29] 0.63492063 0.63095238 0.62845850 0.62450593
 [33] 0.62055336 0.61568627 0.61176471 0.60784314
 [37] 0.60784314 0.60000000 0.60000000 0.59765625
 [41] 0.59375000 0.58593750 0.58203125 0.57421875
 [45] 0.57421875 0.56640625 0.56201550 0.55038760
 [49] 0.53488372 0.51162791 0.50000000 0.49615385
 [53] 0.48659004 0.47892720 0.46360153 0.45593870
 [57] 0.45210728 0.44274809 0.42585551 0.41825095
 [61] 0.41064639 0.39923954 0.39015152 0.38257576
 [65] 0.37735849 0.36981132 0.36226415 0.35471698
 [69] 0.34339623 0.34339623 0.33962264 0.32830189
 [73] 0.31698113 0.29811321 0.29433962 0.28679245
 [77] 0.27924528 0.27067669 0.25563910 0.24436090
 [81] 0.23684211 0.23684211 0.22932331 0.22556391
 [85] 0.21428571 0.19172932 0.18421053 0.17669173
 [89] 0.16541353 0.13909774 0.13909774 0.12781955
 [93] 0.12030075 0.10526316 0.07518797 0.06390977
 [97] 0.06015038 0.05263158 0.04135338 0.01879699
[101] 0.00000000
> roc_obj$specificities
  [1] 0.9179612 0.9205811 0.9237084 0.9253372
  [5] 0.9287434 0.9302885 0.9337176 0.9347722
  [9] 0.9362722 0.9377395 0.9388730 0.9408679
 [13] 0.9424358 0.9448931 0.9469697 0.9498818
 [17] 0.9522910 0.9532357 0.9552098 0.9562353
 [21] 0.9581570 0.9586854 0.9615565 0.9620431
 [25] 0.9639513 0.9654206 0.9664492 0.9669306
 [29] 0.9674570 0.9679517 0.9679517 0.9684747
 [33] 0.9694019 0.9694019 0.9699074 0.9713228
 [37] 0.9727357 0.9736842 0.9737206 0.9737569
 [41] 0.9755985 0.9770009 0.9775126 0.9780119
 [45] 0.9794050 0.9812357 0.9812443 0.9821674
 [49] 0.9826325 0.9826484 0.9831358 0.9840692
 [53] 0.9849932 0.9854678 0.9859347 0.9864069
 [57] 0.9873360 0.9886980 0.9896163 0.9896256
 [61] 0.9900812 0.9900856 0.9905405 0.9905405
 [65] 0.9909950 0.9914491 0.9918992 0.9932524
 [69] 0.9937022 0.9937079 0.9937135 0.9946164
 [73] 0.9955137 0.9964109 0.9968596 0.9968610
 [77] 0.9973118 0.9973154 0.9973166 0.9977639
 [81] 0.9977639 0.9982119 0.9982127 0.9982127
 [85] 0.9982127 0.9982143 0.9986613 0.9986613
 [89] 0.9986613 0.9991079 0.9991079 0.9991083
 [93] 0.9991083 0.9991087 0.9991095 0.9995550
 [97] 1.0000000 1.0000000 1.0000000 1.0000000
[101] 1.0000000

Each point on the ROC curve represents a sensitivity/specificity pair corresponding to a particular decision threshold. That means if the output probability value is more than the threshold than the class associated with it become the called class. For example, if the threshold is set to 0.5 and the output probability of a case being diabetic is 0.75 then the case is considered as a disease.

But when you look at the thresholds in the above examples then it ranges from -inf to +inf including values greater than 1. How can thresholds be greater than 1 when the probability values range from 0 to 1?

Also, I am not able to connect the meaning of -inf and +inf, will it behave the same as max and min-cut numbers of probability like 0 and 1.

Answer 1

But when you look at the thresholds in the above examples then it ranges from -inf to +inf including values greater than 1. How can thresholds be greater than 1 when the probability values range from 0 to 1?

This appears to stem from a typo in your code.

rocobj <- roc(aSAH$outcome, aSAH$s100b)
roc_obj$thresholds

Note the underscore in roc_obj that is not present in rocobj.

Errors like this are why it's best practice to clear your workspace or start a new session before attempting to construct an example.

But when you look at the thresholds in the above examples then it ranges from -inf to +inf including values greater than 1. How can thresholds be greater than 1 when the probability values range from 0 to 1?

First, the data you're using do not vary between (0,1), so it's not a probability.

summary(aSAH$s100b)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  0.030   0.090   0.140   0.247   0.330   2.070 

This is irrelevant, though, because ROC curves don't assume that inputs are probabilities. ROC curves just care about ranking the inputs; as long as the inputs can be ordered, it's fine.

If you print rocobj$thresholds instead of roc_obj$thresholds, you'll see that it has values in the range of the data except for the endpoints -Inf and Inf. The thresholds at the endpoints are -Inf and Inf because ROC curves are defined as monotonic increasing curves from (0,0) to (1,1); you need to have thresholds outside of the range of your data to achieve (0,0) and (1,1); hence -Inf and Inf are used. The interpretation of operating points at (0,0) and (1,1) is no different, because these correspond to the TPR and FPR of naïve models which either never raise the alarm or always raise the alarm.

rocobj$thresholds
 [1]  -Inf 0.035 0.045 0.055 0.065 0.075 0.085 0.095 0.105 0.115 0.125 0.135
[13] 0.145 0.155 0.165 0.175 0.185 0.205 0.225 0.235 0.245 0.255 0.265 0.275
[25] 0.290 0.310 0.325 0.335 0.345 0.365 0.395 0.420 0.435 0.445 0.455 0.465
[37] 0.475 0.485 0.495 0.510 0.540 0.570 0.640 0.705 0.725 0.755 0.795 0.840
[49] 0.910 1.515   Inf