# Questions tagged [auc]

AUC stands for the Area Under the Curve and usually refers to the area under the receiver operator characteristic (ROC) curve.

60 questions
Filter by
Sorted by
Tagged with
3k views

### Is my model any good, based on the diagnostic metric ($R^2$/ AUC/ accuracy/ RMSE etc.) value?

I've fitted my model and am trying to understand whether it's any good. I've calculated the recommended metrics to assess it ($R^2$/ AUC / accuracy / prediction error / etc) but do not know how to ...
122k views

### How to calculate Area Under the Curve (AUC), or the c-statistic, by hand

I am interested in calculating area under the curve (AUC), or the c-statistic, by hand for a binary logistic regression model. For example, in the validation dataset, I have the true value for the ...
376k views

### What does AUC stand for and what is it?

Searched high and low and have not been able to find out what AUC, as in related to prediction, stands for or means.
29k views

### Why is AUC higher for a classifier that is less accurate than for one that is more accurate?

I have two classifiers A: naive Bayesian network B: tree (singly-connected) Bayesian network In terms of accuracy and other measures, A performs comparatively worse than B. However, when I use the R ...
3k views

### Statistical significance (p-value) for comparing two classifiers with respect to (mean) ROC AUC, sensitivity and specificity

I have a test set of 100 cases and two classifiers. I generated predictions and computed ROC AUC, sensitivity and specificity for both classifiers. Question 1: How can I compute p-value to check if ...
2k views

### Compare classifiers based on AUROC or accuracy?

I have a binary classification problem and I experiment different classifiers on it: I want to compare the classifiers. which one is a better measure AUC or accuracy? And why? ...
4k views

### Why AUC =1 even classifier has misclassified half of the samples?

I am using a classifier which returns probabilities. To calculate AUC, I am using pROC R-package. The output probabilities from classifier are: ...
43k views

### What is the difference in what AIC and c-statistic (AUC) actually measure for model fit?

Akaike Information Criterion (AIC) and the c-statistic (area under ROC curve) are two measures of model fit for logistic regression. I am having trouble explaining what is going on when the results of ...
7k views

### How to derive the probabilistic interpretation of the AUC?

Why is the area under the ROC curve the probability that a classifier will rank a randomly chosen "positive" instance (from the retrieved predictions) higher than a randomly chosen "positive" one (...
999 views

### How is a ROCAUC=1.0 possible with imperfect accuracy? [duplicate]

I used sklearn to compute roc_auc_score for a dataset of 72 instances. The accuracy was at 97% (2 misclassifications), but the ROC AUC score was 1.0. How is this ...
3k views

### Did I just invent a Bayesian method for analysis of ROC curves?

Preamble This is a long post. If you're re-reading this, please note that I've revised the question portion, though the background material remains the same. Additionally, I believe that I've devised ...
14k views

### Area under the ROC curve or area under the PR curve for imbalanced data?

I have some doubts about which performance measure to use, area under the ROC curve (TPR as a function of FPR) or area under the precision-recall curve (precision as a function of recall). My data is ...
4k views

### Choosing a classification performance metric for model selection, feature selection, and publication

I have a small, unbalanced data set (70 positive, 30 negative), and I have been playing around with model selection for SVM parameters using BAC (balanced accuracy) and AUC (area under the curve). I ...
10k views

995 views

### Can someone sort me out regarding the calculation of AUC?

I am having some trouble with two different implementations of a classification problem giving different results. Me and my college who did the other implementation has narrowed the problem down to ...
12k views

### AUC and class imbalance in training/test dataset

I just start to learn the Area under the ROC curve (AUC). I am told that AUC is not reflected by data imbalance. I think it means that AUC is insensitive to imbalance in test data, rather than ...
9k views

### Sample size calculation for ROC/AUC analysis

As a background, I am not familiar with stats except on a basic level. I have been tasked with doing some analysis that is out of my comfort zone. I am trying to figure out how to compute necessary ...
28k views

### ROC curve for discrete classifiers like SVM: Why do we still call it a “curve”?, Isn't it just a “point”?

In the discussion : how to generate a roc curve for binary classification, I think that the confusion was that a "binary classifier" (which is any classifier that separates 2 classes) was for Yang ...
3k views

### Why is ROC AUC equivalent to the probability that two randomly-selected samples are correctly ranked? [duplicate]

I found there are two ways to understand what AUC stands for but I couldn't get why these two interpretations are equivalent mathematically. In the first interpretation, AUC is the area under the ...
7k views

### optimizing auc vs logloss in binary classification problems

I am performing a binary classification task where the outcome probability is fair low (aroung 3%). I am trying to decide whether to optimize by AUC or log-loss. As much as I have understood, AUC ...
7k views

### Evaluate Random Forest: OOB vs CV

When we assess the quality of a Random Forest, for example using AUC, is it more appropriate to compute these quantities over the Out of Bag Samples or over the hold out set of cross validation? I ...
3k views

### What to do AFTER nested cross-validation?

I've searched exhaustively on this forum and elsewhere, and have come across a lot of great material. However, I'm ultimately still confused. Here's a basic, concrete example of what I'd like to ...
19k views

### Can AUC-ROC be between 0-0.5?

Can AUC-ROC values be between 0-0.5? Does the model ever output values between 0 and 0.5?
1k views

### Is AUC via CV a good procedure for selecting optimal model?

I'm fitting a logit classifier with LASSO and cross-validation, and struggling to select the optimal model using AUC -instead of the more usual loss like binomial deviance or classification error. I ...
1k views

### ROC/AUC Confidence Interval

For a single ROC curve (with relevant AUC score), how can you calculate the confidence interval? (The data used to generate this ROC/AUC is available) Given my relatively limited background in this ...
527 views

### Evaluating models by Loglogss, AUC, and Accuracy

I am evaluating three models (say, A, B, and C) by three different metrics: Log-loss, AUC, and Accuracy. The results show that Log-loss: C>A>B (B has the best performance in terms of Log-loss) ...
2k views

### How is the optimal probability cut-off in a ROC defined by the R package Epi?

The plot below was created by R package Epi::ROC for a binary classification problem. There is an optimal cut off or threshold for the classification probability at ...
90 views

### When is accuracy score preferred to AUCROC?

I have a binary and balanced dataset. Do I have to see the AUROC as the different trade-offs between the TPR and the FPR and the accuracy as a result with a threshold of 0.5? When is accuracy a ...
36k views

### Area under Precision-Recall Curve (AUC of PR-curve) and Average Precision (AP)

Is Average Precision (AP) the Area under Precision-Recall Curve (AUC of PR-curve) ? EDIT: here is some comment about difference in PR AUC and AP. The AUC is obtained by trapezoidal interpolation ...
16k views

### What are the differences between AUC and F1-score?

F1-score is the harmonic mean of precision and recall. The y-axis of recall is true positive rate (which is also recall). So, sometime classifiers can have low recall but very high AUC, what that ...
2k views

### Optimizing for AUC

AUC is a popular classification evaluation metric. This is a measure of aggregate performance—do any of the standard loss functions (functions of an individual example's label & prediction) ...
11k views

### What is the formula to calculate the area under the ROC curve from a contingency table?

For example, if my table is: ...
3k views

### Statistics for Area under the ROC curve

I have a question regarding statistical evaluation of the AUC. In their paper (http://www.jstor.org/stable/2531595), DeLong et al. describe a method to evaluate AUC curves. (Another good explanation ...
11k views

### How to improve F1 score with skewed classes?

I've a dataset of roughly 40K samples, with 39.6K samples belonging to the target class 0 and 400 to class 1. I've tried several classification algorithms, without too much fine tuning, just to get a ...
330 views

### Does a logistic regression maximizing likelihood necessarily also maximize AUC over linear models?

Given a data set with binary outcomes $y\in\{0,1\}^n$ and some predictor matrix $X\in\mathbb{R}^{n\times p}$, the standard logistic regression model estimates coefficients $\beta_{MLE}$ which maximize ...
8k views

### What is AUC of PR-curve?

I understand that AUC under ROC curve is a classic evaluation measurement for classifiers (which is basically the accuracy). However, when data is imbalanced, PR will be alternative. So, what does the ...
1k views

### pattern of ROC curve and choice of AUC

I am using ROC curves and full AUC values to compare different models, using simulated data. Now I think I am confused with the interpretations of ROC curves and AUC values. Please see the figure ...
4k views

### Relationship between AUC and U Mann-Whitney statistic

Recently I learned about the relationship between Area Under (ROC) Curve and $U$ statistic of the Wilcoxon-Mann-Whitney test. It is supposed to follow the following rule (got it from this nice post on ...
2k views

### Can AUC decrease with additional variables?

I'm fitting a logistic regression model to predict probabilities from a set of variables. I'm comparing two such models, say M1 and ...
417 views

### Statistical Power of ROC/AUC Test with non-IID Samples :: To how many IID Samples are my non-IID Samples Equivalent?

I've been assigned to solve the following problem as part of a serious, biological research project. I think I have a tentative solution, but I'm wondering whether the approach I've picked is the best....
1k views

### Performance Metrics for Imbalanced Classification

I'm trying to fit multiple Stochastic Gradient Descent models to a dataset where the target (binary target, 0 or 1) is very imbalanced, i.e the success rate is about 0.0001. Out of all the models I'...
4k views

### Differences in AUC calculation in R between pROC and AUC

I was comparing the performance of pROC and AUC libraries when performing auc() calculations on random data: ...
1k views

### The distribution of the AUC

I am wondering how the confidence interval for the Area under the Curve statistic (ROC curves) is derived. I have heard that the AUC can be assumed to be normally distributed, but I am looking for a ...
881 views

### AUC with incomplete ROC curve

I am doing an experiments where changing a parameter I am obtaining different number of FalsePositive, FalseNegative... and so on. I am using this parameter tuning as threshold tuning to obtain FPR ...
1k views

### Differences between cross validation and bootstrapping to estimate the standard error of the AUC of a given ROC curve

I know there's been some discussion on differences between CV and bootstrapping for estimating out-of-sample prediction error of a classifier. For example, in here (Differences between cross ...
3k views

### How to distinguish overfitting and underfitting from the ROC AUC curve?

For model selection, one of the metric is (AUC Area Under Curve) which tell us how the models are performing and based on AUC value we can choose the best model. But how to distinguish whether a ...
126 views

### Statistical evidence that the AUC was not overfitted to the model. With N=119, C-stat = 0.81 seems optimistic. Optimism-adjusted?

My data have 119 cases and we did ROC for x (continuous variable) to predict postoperative y (categorical variable) available here, we got a comment from a reviewer asking: Please provide ...
3k views

### AUC vs accuracy for model accuracy evaluation [duplicate]

In R I tried to measure the accuracy by performing a classification analysis using logistic regression analysis. I found that there are two ways to measure accuracy. One is AUC measurement using ROC ...