# Questions tagged [auc]

AUC stands for the Area Under the Curve and usually refers to the area under the receiver operator characteristic (ROC) curve.

485 questions
Filter by
Sorted by
Tagged with
2answers
12k views

### How to interpret 95% confidence interval for Area Under Curve of ROC?

(I am following this paper, from page 47 on http://www.bundesbank.de/Redaktion/EN/Downloads/Tasks/Banking_supervision/working_paper_no_14_studies_on_the_validation_of_internal_rating_systems.pdf?...
1answer
1k views

### Differences between cross validation and bootstrapping to estimate the standard error of the AUC of a given ROC curve

I know there's been some discussion on differences between CV and bootstrapping for estimating out-of-sample prediction error of a classifier. For example, in here (Differences between cross ...
1answer
1k views

### testing equivalence for two independent AUC

First of all, sorry for the "silly" question. I have two AUC, the first one comes from a training set and the other one comes from a validation set. I am using the ...
1answer
5k views

### Using partial AUC as Caret metric for cross-validation?

I'm evaluating a grid of tuning parameters using Caret with metric="ROC" for cross-validation. Is there any simple way to use as metric the area under the curve for an specified interval of the ROC ...
1answer
717 views

### Three way splitting and difference between CV AUC and testing AUC

I have 2000 observations in a dataset with features and a binary-class outcome. I split the dataset into two sets for split sample validation. I use 80% to train the model and internal perform Cross ...
2answers
47k views

### Area under Precision-Recall Curve (AUC of PR-curve) and Average Precision (AP)

Is Average Precision (AP) the Area under Precision-Recall Curve (AUC of PR-curve) ? EDIT: here is some comment about difference in PR AUC and AP. The AUC is obtained by trapezoidal interpolation ...
1answer
6k views

### Area under ROC curve for random forest

Does the area under ROC curve depends on which class is defined as default positive class by the random forest model? I am using ...
1answer
945 views

### Calculating AUC for a GEE

I have used the geeglm package to build a GEE that predicts animal activity (a binary response, active or not) from weather data (e.g., Temperature, a continuous variable). TEMPC <- geeglm(...
1answer
1k views

### How to calculate AUC for any correlation method?

I want to know how to calculate AUC to compare correlation methods. I read this paper http://www.ncbi.nlm.nih.gov/pubmed/23962479 Is there any idea how the authors of above paper have calculated AUC ...
1answer
77 views

### AUC per time is nothing else than the mean?

Using AUC is in vogue and has found his place also in clinical research (example). What I don't understand is AUC per time. For example, if a clinical or psychological parameter is measured over time. ...
1answer
291 views

### So many significant explanatory variables and so small auc

Have you ever seen a model with almost every significant variable and such small auc (area under the ROC curve) ? What might be the cause of it? When I saw summary of a model I thought this model will ...
1answer
953 views

### Random Forest - What training set measure is the best predictor of test set accuracy?

I'm running a random forest model on a training sample in R in order to make predictions on a hidden test set. I'm having difficulty in understanding how I should go about improving my model in order ...
0answers
431 views

### Comparing predictors based on ROC AUC and cross-validation error

I am analysing how well some continuous variables (e.g. weight, height) predict the occurrence of a given disease after surgery. I have computed the area under the curve of the receiver-operator ...
5answers
131k views

### How to calculate Area Under the Curve (AUC), or the c-statistic, by hand

I am interested in calculating area under the curve (AUC), or the c-statistic, by hand for a binary logistic regression model. For example, in the validation dataset, I have the true value for the ...
1answer
3k views

### Optimizing for AUC

AUC is a popular classification evaluation metric. This is a measure of aggregate performance—do any of the standard loss functions (functions of an individual example's label & prediction) ...
1answer
5k views

### Equivalent of AUC (area under the ROC curve) for two variables

I was wondering if there is a way to compute AUC using two variables instead of one as predictors. I got two populations after a follow-up, divided in Cases and Controls according to whether they had ...
1answer
5k views

### How can I calculate AUC using Gini coefficient? [duplicate]

In the Gini Coefficient's Wikipedia page, it is defined as $G= 1 - \frac{\Sigma_{i=1}^n f(y_i)(S_{i-1}+S_i)}{S_n}$ for discrete variables, where $S_i= \Sigma_{j=1}^i f(y_i)y_i$ and $S_0=0$ ($y$ being ...
0answers
876 views

### Calculating two-tailed p-value from z-score for ROC AUC comparison

I am comparing two predictive models by their bootstrapped ROC AUCs with the method originally described by Hanley and McNeil and modified for bootstrapped data by Robin et al. I'm calculating the z-...
1answer
3k views

### Training AUC and CV AUC in Boosted Regression Tree

My question is regarding the differences in the training data AUC score and the cross validation AUC score in boosted regression trees (BRT) built using the gbm.step function in the dismo package. I ...
1answer
2k views

### Is there AUC for neural network?

I am confused about how to calculate AUC for neural network with a softmax classifier. For example, I know that for SVM, we can change the threshold value and determine the AUC. WHat about in neural ...
6answers
405k views

### What does AUC stand for and what is it?

Searched high and low and have not been able to find out what AUC, as in related to prediction, stands for or means.
1answer
2k views

### Is it possible to get confusion matrices from AUC?

When I have one confusion matrix for each cutoff level (from 0.00 to 0.99), I can compute AUC coefficient. It looks like: ...
1answer
2k views

### Connections between $d^\prime$ (d-prime) and AUC (Area Under the ROC Curve); underlying assumptions

In machine learning we may use the area under the ROC curve (often abbreviated AUC, or AUROC) to summarise how well a system can discriminate between two categories. In signal detection theory often ...
1answer
729 views

### ROC / AUC for polynomial Labels

How can I calculate the Area Under Curve for a classifier of a polynomial label in Rapidminer? I could only find a performance operator for binomial labels that provides the AUC value.
1answer
8k views

### R - glmnet - cross validated - AUC [closed]

I have just started working with the glmnet package in R. I have s a dataset which has about 130,000 features and about 32000 rows of data. Here is the code to create the model ...
1answer
11k views

### What is AUC of PR-curve?

I understand that AUC under ROC curve is a classic evaluation measurement for classifiers (which is basically the accuracy). However, when data is imbalanced, PR will be alternative. So, what does the ...
3answers
18k views

### What are the differences between AUC and F1-score?

F1-score is the harmonic mean of precision and recall. The y-axis of recall is true positive rate (which is also recall). So, sometime classifiers can have low recall but very high AUC, what that ...
1answer
1k views

### Significant p value for Mann-Whitney U test but low AUC

How is it possible that for two sample sets I'm getting a low p-value, but also a low AUC value (just below 0.5)? To compute the P-value I'm looking at the second outputted value of the function here ...
2answers
3k views

### what does it mean when out of sample AUC is greater than in sample AUC?

I am fitting a logistic regression model on a data set with about 200,000 observation and 100 features. According to SAS output, the model converged correctly with an in-sample AUC of 0.85. However, ...
1answer
4k views

### Differences in AUC calculation in R between pROC and AUC

I was comparing the performance of pROC and AUC libraries when performing auc() calculations on random data: ...
3answers
31k views

### What is a good AUC for a precision-recall curve?

Because I have a very imbalanced dataset (9% positive outcomes), I decided a precision-recall curve was more appropriate than an ROC curve. I obtained the analogous summary measure of area under the P-...
1answer
189 views

### Binary input to ROC analysis

Im working on assessment of algorithm sensitivity and specificity. I've developed a simulation in order to detect true and false positives and negatives. My intersest is to know if my algorithm is ...
1answer
4k views

### Is it reasonable for a classifier to obtain a high AUC and a low MCC? Or the opposite?

Let's say I have 2 models: 1) High Matthew's correlation coefficient (MCC) score, low area under the curve (AUC) 2) Low MCC, high AUC When I say high and low, I mean relatively to the other model. ...
1answer
1k views

### Differences in AUC calculation between pROC and ROCR

Does anyone know the difference in calculation between these two AUC packages? They get different results when I add in positives with predicted value of 0 (simulating a prob model where many outputs ...
1answer
53 views

### Is Area under curve a composite function

I have some data examples. If I split the data into three parts and the have some scores for each example of the three parts and then calculate individual AUCs for the three parts In the next case, I ...
1answer
2k views

### ROC/AUC Confidence Interval

For a single ROC curve (with relevant AUC score), how can you calculate the confidence interval? (The data used to generate this ROC/AUC is available) Given my relatively limited background in this ...
1answer
2k views

### How can I get cut-off point in multivariated ROC analysis [duplicate]

If I have 1 independent variable (continues) and 1 dependent variable (binary), I can conduct logistic regression and ROC analysis, and I can get a cut-off point of independent variable using ROC ...
1answer
4k views

### R AUC never less than 0.5?

I'm doing some work with random forests in R using the randomForest package, and I've run into something that seems odd to me. Even when the data is completely ...
1answer
9k views

### Sample size calculation for ROC/AUC analysis

As a background, I am not familiar with stats except on a basic level. I have been tasked with doing some analysis that is out of my comfort zone. I am trying to figure out how to compute necessary ...
1answer
431 views

### Statistical Power of ROC/AUC Test with non-IID Samples :: To how many IID Samples are my non-IID Samples Equivalent?

I've been assigned to solve the following problem as part of a serious, biological research project. I think I have a tentative solution, but I'm wondering whether the approach I've picked is the best....
3answers
3k views

### Can AUC decrease with additional variables?

I'm fitting a logistic regression model to predict probabilities from a set of variables. I'm comparing two such models, say M1 and ...
1answer
2k views

### Reverse AUC interpretation

Given a classifier (SVM) classifying in 2 'classes' (+1 or -1) for prediction purposes. It has an AUC score of 0.28, meaning its success rate is lower than just random predictions. If I just do the ...
0answers
154 views

### Comparing AUC vs accuracy [duplicate]

I understand this question has been asked many times however, i am unable to understand the answers well enough and apply to my situation. I have attached 2 screenshots of my model. There are 5 class ...
2answers
4k views

### What to do AFTER nested cross-validation?

I've searched exhaustively on this forum and elsewhere, and have come across a lot of great material. However, I'm ultimately still confused. Here's a basic, concrete example of what I'd like to ...
4answers
396 views

### The value of adding the ROC graph if the AUC is given

I always see in papers that when they want to show how they classifiers performed, they provide ROC graph and the AUC score. Now as far as I know only the AUC tells how well the classifier performed, ...
3answers
15k views

### Area under the ROC curve or area under the PR curve for imbalanced data?

I have some doubts about which performance measure to use, area under the ROC curve (TPR as a function of FPR) or area under the precision-recall curve (precision as a function of recall). My data is ...
3answers
34k views

### Why is AUC higher for a classifier that is less accurate than for one that is more accurate?

I have two classifiers A: naive Bayesian network B: tree (singly-connected) Bayesian network In terms of accuracy and other measures, A performs comparatively worse than B. However, when I use the R ...
1answer
3k views

### Statistics for Area under the ROC curve

I have a question regarding statistical evaluation of the AUC. In their paper (http://www.jstor.org/stable/2531595), DeLong et al. describe a method to evaluate AUC curves. (Another good explanation ...
2answers
421 views

### Predicting class probabilities in regression based on area under the curve

Logistic regression models the log odds. That is for rv $Y$ which is binary logit$(Y=1)=X\beta$. Then with this model, you can estimate the class probabilities and hence prediction or ...
1answer
5k views

### Comparison of two logistic regression models (significant result with anova() but very similar AUCs)

I have compared two logistic regression models using the function anova(mod1,mod2,test="Chisq") in R. The result that I obtained is the following: ...