Timeline for threshold cutoff value from ROC for test set evaluation, do I use the cutoff from test ROC or training ROC
Current License: CC BY-SA 4.0
5 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 17, 2019 at 20:16 | comment | added | Frank Harrell | If you looked at the references I provided you'd see the "why". The central problem with ROC curves for decision making is that they have transposed conditionals as discussed so nicely by Drew Levy in his "Matrix of Confusion" article I reference. Decisions need to be made on the basis of P(outcome) not P(input). And note that any classification you do after doing logistic regression needs to be removed from the logistic regression context, and is usually unnecessary. | |
| Jan 17, 2019 at 16:05 | comment | added | Alexis Drakopoulos | I actually learnt this in final year undergrad at uni, the stats course was quite a mess. We uses ROC for decision making for binary classification using logistic regression. | |
| Jan 17, 2019 at 14:15 | comment | added | Scholar | IMO ROC curves can be useful for decision making, for example if I want to identify cases in a set of cases and controls and want to limit the fraction of false-discoveries to say 0.05, I can do this by selecting the corresponding threshold on a ROC curve. | |
| Jan 17, 2019 at 14:15 | comment | added | Scholar | Yes, logistic regression is used for estimating probabilities but IMO saying that it's not used for classification is overly pedantic, as it is very often used with a classification context in mind. Also, I don't see a reason for why ROC curves shouldn't be used for deciding on a decision threshold. The part in the chapter why supposedly addresses is merely 5 lines long and basically just states ROC curves are bad for decision making, without bothering to explain why. | |
| Jan 17, 2019 at 13:07 | history | answered | Frank Harrell | CC BY-SA 4.0 |