1
$\begingroup$

When we draw a ROC curve, how can we determine the optimal threshold? I think, the point that is closest to (0,1) vector is the best threshold. However, if we cannot calculate the distance, how can we determine the best threshold? Is there any other way to know that?

Also, I think eyeballing is a pretty good way. How do you guys think about the eyeballing method?

$\endgroup$
1
  • $\begingroup$ There are quite a few related questions that are likely duplicates (see the Related links on the right hand side). If you have fixed costs you can tie to to the confusion matrix (costs/benefits for different cells) you can determine an optimal cut point. If not, 'optimal' is not well defined. $\endgroup$
    – Andy W
    Jun 13, 2021 at 15:42

1 Answer 1

4
$\begingroup$

In an ROC analysis, any given threshold for the continuous variable will, in the sample, produce some pair of sensitivity and (1 - specificity) values. Which pair is best is an extra-statistical decision. It depends on the costs of false positives and the benefits of true positives as perceived or assessed for the application or topic in question. An example: “It is better that ten guilty people go free than to convict one innocent person.”

$\endgroup$
1
  • $\begingroup$ Yes, the question always is: "optimal for what?" (+1) The formulas relating costs and benefits for a well calibrated binary model to the probability cutoff are provided here. $\endgroup$
    – EdM
    Jun 13, 2021 at 21:22

Not the answer you're looking for? Browse other questions tagged or ask your own question.