How does one get from ROC curve to selecting the actual decision threshold of a classification model?

Question

Edit to explain how this is different from the suggested duplicate: Reduce Classification Probability Threshold

My question relates to the same topic, but is thoroughly different, so I'm surprised this was flagged as a duplicate. I want to know, specifically, how I go from looking at an ROC curve and identifying the place on the curve that I'm happy with (after considering my specific use case and all the unique ramifications of this choice), to then:

a) Figure out what threshold value $Z$ generated that point on the curve

b) If possible, change my model parameters to now label all $\hat{p}$ >= $Z$ as positive responders. (I.e, is there a "threshold" parameter in, say, scikit-learn with a default value of 0.5, but which I can change to $Z$, or is that final classification step always something that I have to do manually downstream using output probabilities?)

ORIGINAL QUESTION:

I'm well acquainted with the ROC curve and what it represents. I'm wondering what the next practical steps are in selecting a decision threshold for your model once you've plotted an ROC curve. The axes of the curve don't indicate decision thresholds. So you can point to some coordinates on the curve and say "I like this ratio of TPR/FPR," but then how do you find the decision threshold for that point on the plot?

Answer 1

Given only the ROC curve itself, you can't determine thresholds. A set of response and predictor values maps uniquely to an ROC curve, but not the other way around - an ROC curve is defined by the rank order of the predictor values, but their actual numeric values are irrelevant beyond that. Adding 1 to all predictor values, for example, would result in a visually identical ROC curve, but with all the underlying threshold values increased by 1 unit. It's impossible to infer anything about numeric threshold values from an ROC curve alone.

Typically you'll have the underlying data (the response and predictor values) used to generate the ROC curve. Most software packages that generate ROC curves will take those two vectors and generate an array of thresholds and their corresponding TPRs and FPRs, which are used to plot the ROC curve. You'd need to examine this output to find the TPR/FPR you want, and which threshold it corresponds to. Some ROC packages also print threshold values for each point or optimal points directly on the ROC plot.