AUC is an acronym for Area Under Curve.

learn more… | top users | synonyms

1
vote
1answer
29 views

What is a good AUC for a precision-recall curve?

Because I have a very imbalanced dataset (9% positive outcomes), I decided a precision-recall curve was more appropriate than an ROC curve. I obtained the analogous summary measure of area under the ...
0
votes
1answer
23 views

Binary input to ROC analysis

Im working on assessment of algorithm sensitivity and specificity. I've developed a simulation in order to detect true and false positives and negatives. My intersest is to know if my algorithm is ...
2
votes
1answer
64 views

Is it reasonable for a classifier to obtain a high AUC and a low MCC? Or the opposite?

Let's say I have 2 models: 1) High Matthew's correlation coefficient (MCC) score, low area under the curve (AUC) 2) Low MCC, high AUC When I say high and low, I mean relatively to the other model. ...
0
votes
0answers
22 views

Multi-class AUC in Matlab

I would like to compute the area under the ROC-courve (AUC) metric for a classifier with multiple classes. Do you know (reliable) functions for Matlab that implement methods for that, like e.g. in ...
2
votes
1answer
44 views

Differences in AUC calculation between pROC and ROCR

Does anyone know the difference in calculation between these two AUC packages? They get different results when I add in positives with predicted value of 0 (simulating a prob model where many outputs ...
0
votes
0answers
11 views

Is the AUC for dataset (A union B) between the AUC of dataset A and the AUC of dataset B?

Consider you have a binary classifier which you tested on dataset AB=A union B. Assume that the several Area Under the Curve metrics for the three datasets are: AUC(A), AUC(B), and AUC(AB). Without ...
0
votes
1answer
19 views

Is Area under curve a composite function

I have some data examples. If I split the data into three parts and the have some scores for each example of the three parts and then calculate individual AUCs for the three parts In the next case, I ...
1
vote
1answer
36 views

ROC/AUC Confidence Interval

For a single ROC curve (with relevant AUC score), how can you calculate the confidence interval? (The data used to generate this ROC/AUC is available) Given my relatively limited background in this ...
0
votes
1answer
20 views

How can I get cut-off point in multivariated ROC analysis

If I have 1 independent variable (continues) and 1 dependent variable (binary), I can conduct logistic regression and ROC analysis, and I can get a cut-off point of independent variable using ROC ...
1
vote
0answers
57 views

R AUC never less than 0.5?

I'm doing some work with random forests in R using the randomForest package, and I've run into something that seems odd to me. Even when the data is completely ...
2
votes
1answer
82 views

Sample size calculation for ROC/AUC analysis

As a background, I am not familiar with stats except on a basic level. I have been tasked with doing some analysis that is out of my comfort zone. I am trying to figure out how to compute necessary ...
2
votes
1answer
151 views

Statistical Power of ROC/AUC Test with non-IID Samples :: To how many IID Samples are my non-IID Samples Equivalent?

I've been assigned to solve the following problem as part of a serious, biological research project. I think I have a tentative solution, but I'm wondering whether the approach I've picked is the ...
3
votes
2answers
87 views

Can AUC decrease with additional variables?

I'm fitting a logistic regression model to predict probabilities from a set of variables. I'm comparing two such models, say M1 and ...
1
vote
1answer
50 views

Reverse AUC interpretation

Given a classifier (SVM) classifying in 2 'classes' (+1 or -1) for prediction purposes. It has an AUC score of 0.28, meaning its success rate is lower than just random predictions. If I just do the ...
1
vote
0answers
41 views

Comparing AUC vs accuracy

I understand this question has been asked many times however, i am unable to understand the answers well enough and apply to my situation. I have attached 2 screenshots of my model. There are 5 class ...
3
votes
2answers
95 views

What to do AFTER nested cross-validation?

I've searched exhaustively on this forum and elsewhere, and have come across a lot of great material. However, I'm ultimately still confused. Here's a basic, concrete example of what I'd like to ...
0
votes
0answers
30 views

ROC curve and its function beginner

I have 3 features of a signal (example: amplitude, frequency, energy). I want to check which feature is the best to represent that particular signal. That signal is classified into two categories ...
0
votes
0answers
14 views

SVD Down to One Dimension - K=1

I ran an analysis on a very sparse 40K x 40K customer-item rating matrix for recommendations; I first ran SVD on this matrix using many different reduced rank sizes, k=20,30,40... I used the results ...
0
votes
0answers
34 views

How do the ROC cutoffs relate to predictors?

Apologies for this rather simple question, but I haven't been able to find a definition online. What does the ROC cutoffs represent for the AUC package? Specifically, how does it relate to the ...
2
votes
3answers
137 views

The value of adding the ROC graph if the AUC is given

I always see in papers that when they want to show how they classifiers performed, they provide ROC graph and the AUC score. Now as far as I know only the AUC tells how well the classifier performed, ...
2
votes
2answers
142 views

Area under the ROC curve or area under the PR curve for imbalanced data?

I have some doubts about which performance measure to use, area under the ROC curve (TPR as a function of FPR) or area under the precision-recall curve (precision as a function of recall). My data is ...
5
votes
3answers
390 views

Why is AUC higher for a classifier that is less accurate than for one that is more accurate?

I have two classifiers A: naive Bayesian network B: tree (singly-connected) Bayesian network In terms of accuracy and other measures, A performs comparatively worse than B. However, when I use the ...
0
votes
0answers
8 views

help with AUC for PR curve when data has tied and skewed values

I am wondering if there are methods available to calculate AUC for Precision Recall curves when the predicted scores/probs/beliefs(whatever you want to call it) has tied values and could be skewed ...
3
votes
1answer
172 views

Statistics for Area under the ROC curve

I have a question regarding statistical evaluation of the AUC. In their paper (http://www.jstor.org/stable/2531595), DeLong et al. describe a method to evaluate AUC curves. (Another good explanation ...
0
votes
1answer
165 views

Predicting class probabilities in regression based on area under the curve

Logistic regression models the log odds. That is for rv $Y$ which is binary logit$(Y=1)=X\beta$. Then with this model, you can estimate the class probabilities and hence prediction or ...
2
votes
1answer
120 views

Comparison of two logistic regression models (significant result with anova() but very similar AUCs)

I have compared two logistic regression models using the function anova(mod1,mod2,test="Chisq") in R. The result that I obtained is the following: ...
1
vote
0answers
36 views

Calculate AUC of a logistic regression model [duplicate]

I have a data sample of a bank loan history of customers. I have performed logistic regression testing on the sample for finding out how the loan repayment(YES/NO) is dependent on various factors. I ...
1
vote
1answer
50 views

which performance metrics to classify model

I wonder between two performance metrics for classification models: accuracy and area under ROC curve (AUC), which one is to be preferred in which conditions? examples appreciated
0
votes
1answer
103 views

Accuracy and area under ROC curve (AUC)

If we group examples with and without class labels using clustering techniques by treating the class as an ordinary nominal attribute, the resulting clusters can then be used for classifying test ...
1
vote
1answer
176 views

Can someone sort me out regarding the calculation of AUC?

I am having some trouble with two different implementations of a classification problem giving different results. Me and my college who did the other implementation has narrowed the problem down to ...
0
votes
0answers
88 views

AUC is equivalent to a Mann-Whitney U-score, is the basic multiclass AUC related to the Kruskal-Wallis test statistic?

I've read that the area under the ROC curve is equivalent to a Mann-Whitney U-score. Is a multiclass AUC score (which averages the AUC scores for pairs of classes) related to the Kruskal-Wallis test ...
0
votes
0answers
25 views

Sample size when comparing several AUCs

How many individuals do I need to compare 3 AUC which are computed from the same set of patients? Is there a R program or some other program available? There are many programs for computing the ...
3
votes
3answers
204 views

pattern of ROC curve and choice of AUC

I am using ROC curves and full AUC values to compare different models, using simulated data. Now I think I am confused with the interpretations of ROC curves and AUC values. Please see the figure ...
3
votes
2answers
134 views

Estimating ROC/AUC on large data sets?

Plotting an ROC curve of a classifier compared to cases requires that the data set be sorted first on the classifier score. I am in a position where I need to calculate ROC on a large data set very ...
1
vote
1answer
254 views

ROC curves and AUC in simulations to compare models

I am using ROC curves to compare different methods but not sure if I need to re-simulate datasets using different seeds in R in order to reduce the "by-chance" issue for a particular output. Here is a ...
2
votes
3answers
441 views

Choosing a classification performance metric for model selection, feature selection, and publication

I have a small, unbalanced data set (70 positive, 30 negative), and I have been playing around with model selection for SVM parameters using BAC (balanced accuracy) and AUC (area under the curve). I ...
0
votes
1answer
90 views

Test to rank methods by AUCs on various benchmarks

Suppose I have N methods and M benchmarks. I have an AUC statistic (and some other similar statistics) for each combination of method with benchmark. What test should I use to test if one method is ...
1
vote
1answer
189 views

Different score range when calculating area of under curve in ROC curves

I have two classifiers which try to classify the same data sets. In order to check the efficiency of the classifiers I intend to plot the curves and calculate the AUC value. The concern is that one of ...
0
votes
0answers
151 views

How to use pROC package in R

I have a pool of factor numbers and a classifier which determines if a factor plays role in a disease or not. So the test result is "yes" or "no" which shows whether the factor involves in the disease ...
1
vote
1answer
223 views

Calculate LOO-AUC values using glmnet

I have a matrix (x) containing 55 samples (rows) and 10000 independent variables (columns). The observations are binary, healthy or ill {0,1} (y). I want to perform leave one out cross-validation and ...
1
vote
1answer
389 views

Calculation of AUC value from ROC Curve

Is there any tool that can calculate the AUC value from a ROC curve if I already know how many samples are true positive, true negative, false positive, false negative out of 500 samples? Specificity ...
0
votes
0answers
47 views

Tool for calculating AUC Value [duplicate]

Is there any tool that can calculate the AUC value from a ROC curve if I already have true positive, true negative, false positive, false negative values.
1
vote
1answer
250 views

How can I validate a logistic regression model using averaged parameter estimates?

Let me say thanks in advance. I'm working with a set of data that contains reported coyote sightings. I use 2/3 of the data for model calibration along with an equal number of pseudo absences. I ...
2
votes
3answers
188 views

How can I tell if my binary classifier is any good?

Say I have a data set with 10,000 rows and the target is a binary variable with 1500 positives (1's) and 8500 negatives (0's). I run a model and get predictions on the 0-1 interval. My question is ...
2
votes
2answers
137 views

Alternating AUC curve. What does it mean?

Why do I see my ROC curve crossing the line from (0,0) to (1,1) (i.e. the 0-1 line)? I have the following test data as a tab-separated testdata.txt file. Running my R code (given below) multiple ...
2
votes
1answer
143 views

AUC for more than two groups?

Standard ROC curves look at how setting various thresholds on a continuous measure can be used to predict a two-level ordinal outcome (example: antibody level -> (not sick, sick) ). This can then be ...
2
votes
0answers
126 views

Role of coefficients in model selection for logistic regression

I have a model that I am using to predict mortality and it gives me an AUC of 0.799. The R code that I am using would look something like this: ...
1
vote
0answers
100 views

Using AUC to compare logistic lasso and elastic net

I've seen this question answered here but I do not understand the answer. Harrell recommends using deviance based measures. David Hand (referenced in the thread) says that that the AUC is ...
5
votes
2answers
284 views

Compare classifiers based on AUROC or accuracy?

I have a binary classification problem and I experiment different classifiers on it: I want to compare the classifiers. which one is a better measure AUC or accuracy? And why? ...
9
votes
2answers
6k views

What is the difference in what AIC and c-statistic (AUC) actually measure for model fit?

Akaike Information Criterion (AIC) and the c-statistic (area under ROC curve) are two measures of model fit for logistic regression. I am having trouble explaining what is going on when the results of ...