# Tag Info

### Can AUC-ROC be between 0-0.5?

I am sorry, but these answers are dangerously wrong. No, you cannot just flip AUC after you see the data. Imagine you are buying stocks, and you always bought the wrong one, but you said to yourself, ...
• 7,629
Accepted

### Can AUC-ROC be between 0-0.5?

A perfect predictor gives an AUC-ROC score of 1, a predictor which makes random guesses has an AUC-ROC score of 0.5. If you get a score of 0 that means the classifier is perfectly incorrect, it is ...
• 4,049

### ROC vs precision-and-recall curves

There is a lot of misunderstanding about evaluation. Part of this comes from the Machine Learning approach of trying to optimize algorithms on datasets, with no real interest in the data. In a ...
• 1,461

### What does it mean that AUC is a semi-proper scoring rule?

Let's start with an example. Say Alice is a track coach and wants to pick an athlete to represent the team in an upcoming sporting event, a 200m sprint. Naturally she wants to pick the fastest runner. ...
• 44.4k
Accepted

### What is the formula to calculate the area under the ROC curve from a contingency table?

In the general case: you can't The ROC curve shows how sensitivity and specificity varies at every possible threshold. A contingency table has been calculated at a single threshold and information ...
• 3,674

### Is the Dice coefficient the same as accuracy?

The Dice coefficient (also known as Dice similarity index) is the same as the F1 score, but it's not the same as accuracy. The main difference might be the fact that accuracy takes into account true ...
• 461

### ROC vs Precision-recall curves on imbalanced dataset

First, the claim on the Kaggle post is bogus. The paper they reference, "The Relationship Between Precision-Recall and ROC Curves", never claims that PR AUC is better than ROC AUC. They simply compare ...
• 3,674

### How to choose between ROC AUC and F1 score?

None of the measures listed here are proper accuracy scoring rules, i.e., rules that are optimized by a correct model. Consider the Brier score and log-likelihood-based measures such as pseudo $R^2$. ...
• 92.9k

### How to choose between ROC AUC and F1 score?

Calculation formula： Precision: TP/(TP+FP) Recall: TP/(TP+FN) F1-score： 2/(1/P+1/R) ROC/AUC： TPR=TP/(TP+FN), FPR=FP/(FP+TN) ROC / AUC is the same criteria and the PR (Precision-Recall) curve (F1-...
• 516
Accepted

### Comparison of two models when the ROC curves cross each other

A ROC curve visualizes TPR and FPR for all possible thresholds. If you plot two ROC curves 'A' and 'B' and they do not cross each other, then one of your classifiers is clearly performing better, ...
• 2,041

### Where in the ROC curve does it tell you what the threshold is?

Each (FPR, TPR) point on a ROC curve is associated with a threshold. However, the thresholds are not typically drawn on the curve itself. It is possible to reveal them, either adding extra annotation ...
• 3,674

### What is the difference between GINI and AUC curve interpretation?

The Gini Coefficient is the summary statistic of the Cumulative Accuracy Profile (CAP) chart. It is calculated as the quotient of the area which the CAP curve and diagonal enclose and the ...
• 361

### How to derive the probabilistic interpretation of the AUC?

From Measuring classifier performance: a coherent alternative to the area under the ROC curve : First thing, let's try to define the area under the ROC curve formally. Some assumptions and ...
• 419
Accepted

A threshold isn't trained with the model because logistic regression isn't a classifier (cf., Why isn't Logistic Regression called Logistic Classification?). It is a model to estimate the parameter, $... 20 votes ### What does AUC stand for and what is it? The answers in this forum are great and I come back here often for reference. However, one thing was always missing. From @Frank's answer, we see the interpretation of AUC as the probability that a ... • 2,540 19 votes ### How to derive the probabilistic interpretation of the AUC? @alebu's answer is great. But its notation is nonstandard and uses 0 for the positive class and 1 for the negative class. Below are the results for the standard notation (0 for the negative class and ... • 956 19 votes Accepted ### AUPRC vs. AUC-ROC? ROC AUC is the area under the curve where x is false positive rate (FPR) and y is true positive rate (TPR). PR AUC is the area under the curve where x is recall and y is precision. recall = TPR = ... • 5,740 19 votes ### Determine how good an AUC is (Area under the Curve of ROC) From the comments: Calimo: If you are a trader and you can get an AUC of 0.501 in predicting future financial transactions, you're the richest man in the world. If you are a CPU engineer and your ... 18 votes ### How to determine the optimal threshold for a classifier and generate ROC curve? The choice of a threshold depends on the importance of TPR and FPR classification problem. For example, if your classifier will decide which criminal suspects will receive a death sentence, false ... • 728 18 votes Accepted ### Statistical significance (p-value) for comparing two classifiers with respect to (mean) ROC AUC, sensitivity and specificity Wojtek J. Krzanowski and David J. Hand ROC Curves for Continuous Data (2009) is a great reference for all things related to ROC curves. It collects together a number of results in what is a ... • 91.4k 17 votes Accepted ### Is AUC the probability of correctly classifying a randomly selected instance from each class? The quotation is slightly incorrect. The correct statement is that ROC AUC is the probability a randomly-chosen positive example is ranked more highly than a randomly-chosen negative example. This is ... • 91.4k 17 votes Accepted ### AUC and class imbalance in training/test dataset It depends how you mean the word sensitive. The ROC AUC is sensitive to class imbalance in the sense that when there is a minority class, you typically define this as the positive class and it will ... • 3,171 17 votes ### ROC vs Precision-recall curves on imbalanced dataset Your example is definitely correct. However, I think in the context of Kaggle competition / real life application, a skewed dataset usually means a dataset with much less positive samples than ... • 171 17 votes ### Can AUC-ROC be between 0-0.5? They can, if the system you're analyzing performs below chance level. Trivially, you could easily construct a classifier with 0 AUC by having it always answer opposite to the truth. In practice of ... • 7,559 17 votes Accepted ### What is the origin of the "receiver operating characteristic" (ROC) terminology? The earliest book reference that I know of is Woodward, P. M. (1953). Probability and information theory with applications to radar. London: Pergamon Press. but the concept, which was developed ... • 46.9k 17 votes Accepted ### Dichotomizing continuous variables at their optimal cut-off for clinical interpretation Dichotomizing a continuous covariate is ill-advised, as has been noted by other users. One strategy I employ is to rescale the predictor to something more reasonable. 1 mmHg may not be a very ... • 36.9k 16 votes ### How to determine the optimal threshold for a classifier and generate ROC curve? Choose the point closest to the top left corner of your ROC space. Now the threshold used to generate this point should be the optimal one. • 269 16 votes ### Area under the ROC curve when there is imbalance: is there a problem, and if not, why does this rumor exist? This is actually a very simple issue. The area under the ROC curve (AUROC) equals the Wilcoxon-Mann-Whitney-Somers concordance probability, a$U$-statistic, i.e., take all possible pairs of an ... • 92.9k 15 votes ### Is up- or down-sampling imbalanced data actually that effective? Why? The short answer appears to be Yes: there is some evidence that upsampling of the minority class and/or downsampling of the majority class in a training set can somewhat improve out-of-sample AUC (... • 12.6k 15 votes ### ROC vs precision-and-recall curves TL;DR$AUC_{PvR}$highlights the amount of False Positives relative to the class size, whereas$AUC_{ROC}\$ better reflects the total amount of False Positives independent of in which class they come ...
• 289

Only top scored, non community-wiki answers of a minimum length are eligible