Timeline for Choosing a Cut-Off Value from an ROC Curve for a Cross Validated Dataset
Current License: CC BY-SA 4.0
7 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Aug 5, 2022 at 13:16 | answer | added | Frank Harrell | timeline score: 1 | |
| Aug 5, 2022 at 12:04 | answer | added | cbeleites | timeline score: 2 | |
| Aug 5, 2022 at 8:22 | comment | added | MauM99 | Frank is right. Yes I also want to use the AUC, as well as the accuracy of the the model in total, in order to compare it with another method. And the accuracy is dependent of the cut-off-value. Therefore, because specifity and sensintivity differs with the folds, should I pick a value based on a qualitative decision for every fold? | |
| Aug 5, 2022 at 5:04 | comment | added | frank | @Dave But the OP is interested in accuracy. | |
| Aug 5, 2022 at 5:01 | comment | added | Dave | @frank A major point behind ROC curves is that you can set the cutoff wherever you want, and the classifications will be different for different cutoff values, resulting in different balances of sensitivity and specificity. In the extreme, we can achieve perfect sensitivity or specificity by sacrificing the other. | |
| Aug 5, 2022 at 4:57 | comment | added | frank | You want to classify data into two classes, say 0 and 1, and you have a logistic regression model that gives you, for each observation, the probability that this observation should be classified as 1. So you should assign the label 1 whenever this probability is greater than 1/2. What do you need a Cut-Off value for? Maybe I don't understand correctly what you refer to by "Cut-Off value". | |
| Aug 5, 2022 at 1:07 | history | asked | MauM99 | CC BY-SA 4.0 |