Search Results

ROC curves provide no actionable insights. They have become obligatory without researchers examining the benefits. They have a very large ink:information ratio. … but can be more easily understood if you don't draw the ROC) and the calibration curve. …

It is good that they disagree with proper scoring (log-likelihood; logarithmic scoring rule; Brier score) rules and the $c$-index (a semi-proper scoring rule - area under ROC curve; concordance probability …

None of the measures listed here are proper accuracy scoring rules, i.e., rules that are optimized by a correct model. Consider the Brier score and log-likelihood-based measures such as pseudo $R^2$. …

I think you are interested in predictive discrimination, for which a generalized $R^2$ measure, supplemented by $c$-index (ROC area) may be more appropriate. …

The area under the ROC curve (AUROC) equals the Wilcoxon-Mann-Whitney-Somers concordance probability, a $U$-statistic, i.e., take all possible pairs of an observation with Y=0 and an observation with Y … Likewise every point on the ROC curve conditions on Y so the entire curve is conditional on Y. Each point is made up of probabilities like $\Pr(X > x | Y=y)$ ($y=0$ for x-axis, $y=1$ for y-axis). …

A good measure is Somers' Dxy rank correlation, a generalization of ROC area for ordinal or continuous Y. …

ROC curves and some of the other measures given in the discussion don't help. …

And an ROC curve is irrelevant to this issue. …

I would use the quadratic proper scoring rule known as the Brier score, or the concordance probability (area under ROC curve in the binary $Y$ case). …

Concordance probability ($c$-index; ROC area) is a measure of pure discrimination. …

To make an optimal decision you need to know all relevant data about an individual (used to estimate the probability of an outcome), and the utility (cost, loss function) of making each decision. Sen …

One is the $c$-index or ROC area as others have described. This has an interpretation that is simpler than thinking about an ROC curve, and is a measure of pure predictive discrimination. …

$D_{xy} = 2(C - \frac{1}{2})$ where $C$ is the generalized ROC area (concordance probability). Intercept and Slope pertain to the calibration curve on the logit scale. …

ROC curves are not informative in 99% of the cases I've seen over the past few years. They seem to be thought of as obligatory by many statisticians and even more machine learning practitioners. … ROC curves cannot be used to find optimum tradeoffs except in very special cases where users of a decision rule abdicate their loss (cost; utility) function to the analyst. …

You are using a non-optimum optimality criterion (the $c$-index; generalized ROC area) whereas the most sensitive criterion will be likelihood-based Your total sample size is too low by perhaps a factor …