361
votes
Accepted
What does AUC stand for and what is it?
Abbreviations
AUC = Area Under the Curve.
AUROC = Area Under the Receiver Operating Characteristic curve.
AUC is used most of the time to mean AUROC, which is a bad practice since as Marc Claesen ...
- 42.7k
195
votes
Accepted
Understanding ROC curve
I'm not sure I got the question, but since the title asks for explaining ROC curves, I'll try.
ROC Curves are used to see how well your classifier can separate positive and negative examples and to ...
- 8,257
136
votes
Accepted
How to calculate Area Under the Curve (AUC), or the c-statistic, by hand
I would recommend Hanley’s & McNeil’s 1982 paper ‘The meaning and use of the area under a receiver operating characteristic (ROC) curve’.
Example
They have the following table of disease status ...
- 5,212
73
votes
What does AUC stand for and what is it?
Although I'm a bit late to the party, but here's my 5 cents. @FranckDernoncourt (+1) already mentioned possible interpretations of AUC ROC, and my favorite one is the first on his list (I use ...
- 8,257
57
votes
What does AUC stand for and what is it?
Important considerations are not included in any of these discussions. The procedures discussed above invite inappropriate thresholding and utilize improper accuracy scoring rules (proportions) that ...
- 75.5k
37
votes
ROC vs precision-and-recall curves
Here are the conclusions from a paper by Davis & Goadrich explaining the relationship between ROC and PR space. They answer the first two questions:
First, for any dataset, the ROC curve and PR ...
- 1,521
36
votes
Can AUC-ROC be between 0-0.5?
I am sorry, but these answers are dangerously wrong. No, you cannot just flip AUC after you see the data. Imagine you are buying stocks, and you always bought the wrong one, but you said to yourself, ...
- 6,126
34
votes
How to calculate Area Under the Curve (AUC), or the c-statistic, by hand
Have a look at this question: Understanding ROC curve
Here's how to build a ROC curve (from that question):
Drawing ROC curve
given a data set processed by your ranking classifier
rank test ...
- 8,257
34
votes
Accepted
Can AUC-ROC be between 0-0.5?
A perfect predictor gives an AUC-ROC score of 1, a predictor which makes random guesses has an AUC-ROC score of 0.5.
If you get a score of 0 that means the classifier is perfectly incorrect, it is ...
- 3,679
33
votes
Accepted
Area under the ROC curve or area under the PR curve for imbalanced data?
The question is quite vague so I am going to assume you want to choose an appropriate performance measure to compare different models. For a good overview of the key differences between ROC and PR ...
- 17.5k
32
votes
Accepted
Why is AUC higher for a classifier that is less accurate than for one that is more accurate?
Improper scoring rules such as proportion classified correctly, sensitivity, and specificity are not only arbitrary (in choice of threshold) but are improper, i.e., they have the property that ...
- 75.5k
30
votes
Is the Dice coefficient the same as accuracy?
The Dice coefficient (also known as Dice similarity index) is the same as the F1 score, but it's not the same as accuracy. The main difference might be the fact that accuracy takes into account true ...
- 421
29
votes
What does it mean that AUC is a semi-proper scoring rule?
Let's start with an example. Say Alice is a track coach and wants to pick an athlete to represent the team in an upcoming sporting event, a 200m sprint. Naturally she wants to pick the fastest runner.
...
- 35.1k
29
votes
Accepted
What is the formula to calculate the area under the ROC curve from a contingency table?
In the general case: you can't
The ROC curve shows how sensitivity and specificity varies at every possible threshold. A contingency table has been calculated at a single threshold and information ...
- 2,899
27
votes
ROC vs precision-and-recall curves
There is a lot of misunderstanding about evaluation. Part of this comes from the Machine Learning approach of trying to optimize algorithms on datasets, with no real interest in the data.
In a ...
- 1,336
27
votes
Difference between regression analysis and curve fitting
I doubt that there is a clear and consistent distinction across statistically minded sciences and fields between regression and curve-fitting.
Regression without qualification implies linear ...
- 48.9k
27
votes
Accepted
What is the name of this chart showing false and true positive rates and how is it generated?
The plot is ROC curve and the (False Positive Rate, True Positive Rate) points are calculated for different thresholds. Assuming you have an uniform utility function, the optimal threshold value is ...
- 938
26
votes
Accepted
Why is ROC AUC equivalent to the probability that two randomly-selected samples are correctly ranked?
It's easy to see once you obtained a closed-form formula for AUC.
Since we have finite number of samples $\{(x_i, y_i)\}_{i=1}^N$, we'll have finite number of points on the ROC curve. We do linear ...
- 2,581
26
votes
Accepted
Is the Dice coefficient the same as accuracy?
These are not the same thing and they are often used in different contexts. The Dice score is often used to quantify the performance of image segmentation methods. There you annotate some ground truth ...
- 3,571
23
votes
What is the convex hull in ROC curve?
The paper gives the following definition, which is pretty much a constructive one:
Linear interpolation is used between adjacent points.
No point lies above the final curve.
For any pair of ...
- 260k
23
votes
Accepted
How to determine the optimal threshold for a classifier and generate ROC curve?
Use the SVM classifier to classify a set of annotated examples, and "one point" on the ROC space based on one prediction of the examples can be identified. Suppose the number of examples is ...
- 818
23
votes
Accepted
How to interpret a ROC curve?
When you do logistic regression, you are given two classes coded as $1$ and $0$. Now, you compute probabilities that given some explanatory varialbes an individual belongs to the class coded as $1$. ...
- 2,282
21
votes
How to choose between ROC AUC and F1 score?
Calculation formula：
Precision TP/(TP+FP)
Recall: TP/(TP+FN)
F1-score： 2/(1/P+1/R)
ROC/AUC： TPR=TP/(TP+FN), FPR=FP/(FP+TN)
ROC / AUC is the same criteria and the PR (Precision-Recall) curve (F1-...
- 488
21
votes
ROC vs Precision-recall curves on imbalanced dataset
First, the claim on the Kaggle post is bogus. The paper they reference, "The Relationship Between Precision-Recall and ROC Curves", never claims that PR AUC is better than ROC AUC. They simply compare ...
- 2,899
21
votes
Accepted
Comparison of two models when the ROC curves cross each other
A ROC curve visualizes TPR and FPR for all possible thresholds.
If you plot two ROC curves 'A' and 'B' and they do not cross each other, then one of your classifiers is clearly performing better, ...
- 1,712
20
votes
What is the difference between GINI and AUC curve interpretation?
The Gini Coefficient is the summary statistic of the Cumulative Accuracy Profile (CAP) chart. It is calculated as the quotient of the area which the CAP curve and diagonal enclose and the ...
- 331
20
votes
What is the name of this chart showing false and true positive rates and how is it generated?
To generate ROC curves (= Receiver Operating Characteristic curves):
Assume we have a probabilistic, binary classifier such as logistic regression. Before presenting the ROC curve, the concept of ...
- 42.7k
20
votes
Accepted
Philosophical question on logistic regression: why isn't the optimal threshold value trained?
A threshold isn't trained with the model because logistic regression isn't a classifier (cf., Why isn't Logistic Regression called Logistic Classification?). It is a model to estimate the parameter, $...
19
votes
Why is AUC higher for a classifier that is less accurate than for one that is more accurate?
Why is the AUC for A better than B, when B "seems" to outperform A with respect to accuracy?
Accuracy is computed at the threshold value of 0.5. While AUC is computed by adding all the "...
- 6,474
19
votes
Average ROC for repeated 10-fold cross validation with probability estimates
From your description it seems to make perfect sense: not only you may calculate the mean ROC curve, but also the variance around it to build confidence intervals. It should give you the idea of how ...
- 8,257
Only top scored, non community-wiki answers of a minimum length are eligible