Hot answers tagged

361 votes
Accepted

What does AUC stand for and what is it?

Abbreviations AUC = Area Under the Curve. AUROC = Area Under the Receiver Operating Characteristic curve. AUC is used most of the time to mean AUROC, which is a bad practice since as Marc Claesen ...
user avatar
195 votes
Accepted

Understanding ROC curve

I'm not sure I got the question, but since the title asks for explaining ROC curves, I'll try. ROC Curves are used to see how well your classifier can separate positive and negative examples and to ...
user avatar
136 votes
Accepted

How to calculate Area Under the Curve (AUC), or the c-statistic, by hand

I would recommend Hanley’s & McNeil’s 1982 paper ‘The meaning and use of the area under a receiver operating characteristic (ROC) curve’. Example They have the following table of disease status ...
user avatar
73 votes

What does AUC stand for and what is it?

Although I'm a bit late to the party, but here's my 5 cents. @FranckDernoncourt (+1) already mentioned possible interpretations of AUC ROC, and my favorite one is the first on his list (I use ...
user avatar
57 votes

What does AUC stand for and what is it?

Important considerations are not included in any of these discussions. The procedures discussed above invite inappropriate thresholding and utilize improper accuracy scoring rules (proportions) that ...
user avatar
37 votes

ROC vs precision-and-recall curves

Here are the conclusions from a paper by Davis & Goadrich explaining the relationship between ROC and PR space. They answer the first two questions: First, for any dataset, the ROC curve and PR ...
user avatar
  • 1,521
36 votes

Can AUC-ROC be between 0-0.5?

I am sorry, but these answers are dangerously wrong. No, you cannot just flip AUC after you see the data. Imagine you are buying stocks, and you always bought the wrong one, but you said to yourself, ...
user avatar
  • 6,126
34 votes

How to calculate Area Under the Curve (AUC), or the c-statistic, by hand

Have a look at this question: Understanding ROC curve Here's how to build a ROC curve (from that question): Drawing ROC curve given a data set processed by your ranking classifier rank test ...
user avatar
34 votes
Accepted

Can AUC-ROC be between 0-0.5?

A perfect predictor gives an AUC-ROC score of 1, a predictor which makes random guesses has an AUC-ROC score of 0.5. If you get a score of 0 that means the classifier is perfectly incorrect, it is ...
user avatar
  • 3,679
33 votes
Accepted

Area under the ROC curve or area under the PR curve for imbalanced data?

The question is quite vague so I am going to assume you want to choose an appropriate performance measure to compare different models. For a good overview of the key differences between ROC and PR ...
user avatar
  • 17.5k
32 votes
Accepted

Why is AUC higher for a classifier that is less accurate than for one that is more accurate?

Improper scoring rules such as proportion classified correctly, sensitivity, and specificity are not only arbitrary (in choice of threshold) but are improper, i.e., they have the property that ...
user avatar
30 votes

Is the Dice coefficient the same as accuracy?

The Dice coefficient (also known as Dice similarity index) is the same as the F1 score, but it's not the same as accuracy. The main difference might be the fact that accuracy takes into account true ...
user avatar
  • 421
29 votes

What does it mean that AUC is a semi-proper scoring rule?

Let's start with an example. Say Alice is a track coach and wants to pick an athlete to represent the team in an upcoming sporting event, a 200m sprint. Naturally she wants to pick the fastest runner. ...
user avatar
  • 35.1k
29 votes
Accepted

What is the formula to calculate the area under the ROC curve from a contingency table?

In the general case: you can't The ROC curve shows how sensitivity and specificity varies at every possible threshold. A contingency table has been calculated at a single threshold and information ...
user avatar
  • 2,899
27 votes

ROC vs precision-and-recall curves

There is a lot of misunderstanding about evaluation. Part of this comes from the Machine Learning approach of trying to optimize algorithms on datasets, with no real interest in the data. In a ...
user avatar
27 votes

Difference between regression analysis and curve fitting

I doubt that there is a clear and consistent distinction across statistically minded sciences and fields between regression and curve-fitting. Regression without qualification implies linear ...
user avatar
  • 48.9k
27 votes
Accepted

What is the name of this chart showing false and true positive rates and how is it generated?

The plot is ROC curve and the (False Positive Rate, True Positive Rate) points are calculated for different thresholds. Assuming you have an uniform utility function, the optimal threshold value is ...
user avatar
  • 938
26 votes
Accepted

Why is ROC AUC equivalent to the probability that two randomly-selected samples are correctly ranked?

It's easy to see once you obtained a closed-form formula for AUC. Since we have finite number of samples $\{(x_i, y_i)\}_{i=1}^N$, we'll have finite number of points on the ROC curve. We do linear ...
user avatar
26 votes
Accepted

Is the Dice coefficient the same as accuracy?

These are not the same thing and they are often used in different contexts. The Dice score is often used to quantify the performance of image segmentation methods. There you annotate some ground truth ...
user avatar
  • 3,571
23 votes

What is the convex hull in ROC curve?

The paper gives the following definition, which is pretty much a constructive one: Linear interpolation is used between adjacent points. No point lies above the final curve. For any pair of ...
user avatar
  • 260k
23 votes
Accepted

How to determine the optimal threshold for a classifier and generate ROC curve?

Use the SVM classifier to classify a set of annotated examples, and "one point" on the ROC space based on one prediction of the examples can be identified. Suppose the number of examples is ...
user avatar
  • 818
23 votes
Accepted

How to interpret a ROC curve?

When you do logistic regression, you are given two classes coded as $1$ and $0$. Now, you compute probabilities that given some explanatory varialbes an individual belongs to the class coded as $1$. ...
user avatar
  • 2,282
21 votes

How to choose between ROC AUC and F1 score?

Calculation formula: Precision TP/(TP+FP) Recall: TP/(TP+FN) F1-score: 2/(1/P+1/R) ROC/AUC: TPR=TP/(TP+FN), FPR=FP/(FP+TN) ROC / AUC is the same criteria and the PR (Precision-Recall) curve (F1-...
user avatar
  • 488
21 votes

ROC vs Precision-recall curves on imbalanced dataset

First, the claim on the Kaggle post is bogus. The paper they reference, "The Relationship Between Precision-Recall and ROC Curves", never claims that PR AUC is better than ROC AUC. They simply compare ...
user avatar
  • 2,899
21 votes
Accepted

Comparison of two models when the ROC curves cross each other

A ROC curve visualizes TPR and FPR for all possible thresholds. If you plot two ROC curves 'A' and 'B' and they do not cross each other, then one of your classifiers is clearly performing better, ...
user avatar
20 votes

What is the difference between GINI and AUC curve interpretation?

The Gini Coefficient is the summary statistic of the Cumulative Accuracy Profile (CAP) chart. It is calculated as the quotient of the area which the CAP curve and diagonal enclose and the ...
user avatar
  • 331
20 votes

What is the name of this chart showing false and true positive rates and how is it generated?

To generate ROC curves (= Receiver Operating Characteristic curves): Assume we have a probabilistic, binary classifier such as logistic regression. Before presenting the ROC curve, the concept of ...
user avatar
20 votes
Accepted

Philosophical question on logistic regression: why isn't the optimal threshold value trained?

A threshold isn't trained with the model because logistic regression isn't a classifier (cf., Why isn't Logistic Regression called Logistic Classification?). It is a model to estimate the parameter, $...
user avatar
19 votes

Why is AUC higher for a classifier that is less accurate than for one that is more accurate?

Why is the AUC for A better than B, when B "seems" to outperform A with respect to accuracy? Accuracy is computed at the threshold value of 0.5. While AUC is computed by adding all the "...
user avatar
  • 6,474
19 votes

Average ROC for repeated 10-fold cross validation with probability estimates

From your description it seems to make perfect sense: not only you may calculate the mean ROC curve, but also the variance around it to build confidence intervals. It should give you the idea of how ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible