Timeline for Choosing between logistic and discriminant
Current License: CC BY-SA 3.0
6 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 24, 2016 at 19:18 | comment | added | Frank Harrell | Well put. My point is that to get an interpretable (if insensitive) measure of predictive discrimination the curve doesn't help at all, but the area under the curve ($c$-index), which can be computed quickly and easily using rank correlation/Wilcoxon test ideas, does help. It is unfortunate that an idea so clean as concordance happens to coincide with the area under such a silly curve. | |
| Jan 24, 2016 at 14:41 | comment | added | nootodis | When you say that you "use" an ROC curve, is it for understanding the model's discriminative power after a model is chosen? Not to be used to make any type of model calibration. I see it used a lot for choosing a better threshold value, but from what I read from you, it is better to keep the model in context of it's probability then assigning it a threshold. | |
| Jan 24, 2016 at 14:36 | vote | accept | nootodis | ||
| Jan 23, 2016 at 19:24 | comment | added | Frank Harrell | IMHO ROC curves tell you nothing useful of any sort. And I don't want to know about 'ability to classify'; I want to know about ability to predict, and I don't want to use a method such as ROC that invites analysts to use thresholds. For model performance I want to use a proper accuracy score plus make a high resolution nonparametric calibration curve. I do use ROC areas ($c$-index; concordance probability) because it is an interpretable (if insensitive) measure of pure predictive discrimination. | |
| Jan 23, 2016 at 19:06 | comment | added | nootodis | Appreciate the link. If I understand correctly from your paper, ROC curves only measure the ability of a model to classify what group the data belongs to, but it's lack of ability to tell how reliable a model is, therefore it is not a good way of measuring model performance. | |
| Jan 22, 2016 at 22:04 | history | answered | Frank Harrell | CC BY-SA 3.0 |