AUC is an acronym for Area Under Curve.

learn more… | top users | synonyms

0
votes
1answer
10 views

How can I get cut-off point in multivariated ROC analysis

If I have 1 independent variable (continues) and 1 dependent variable (binary), I can conduct logistic regression and ROC analysis, and I can get a cut-off point of independent variable using ROC ...
1
vote
0answers
40 views

R AUC never less than 0.5?

I'm doing some work with random forests in R using the randomForest package, and I've run into something that seems odd to me. Even when the data is completely ...
2
votes
1answer
36 views

Sample size calculation for ROC/AUC analysis

As a background, I am not familiar with stats except on a basic level. I have been tasked with doing some analysis that is out of my comfort zone. I am trying to figure out how to compute necessary ...
2
votes
1answer
133 views

Statistical Power of ROC/AUC Test with non-IID Samples :: To how many IID Samples are my non-IID Samples Equivalent?

I've been assigned to solve the following problem as part of a serious, biological research project. I think I have a tentative solution, but I'm wondering whether the approach I've picked is the ...
3
votes
2answers
80 views

Can AUC decrease with additional variables?

I'm fitting a logistic regression model to predict probabilities from a set of variables. I'm comparing two such models, say M1 and ...
1
vote
1answer
47 views

Reverse AUC interpretation

Given a classifier (SVM) classifying in 2 'classes' (+1 or -1) for prediction purposes. It has an AUC score of 0.28, meaning its success rate is lower than just random predictions. If I just do the ...
1
vote
0answers
39 views

Comparing AUC vs accuracy

I understand this question has been asked many times however, i am unable to understand the answers well enough and apply to my situation. I have attached 2 screenshots of my model. There are 5 class ...
3
votes
2answers
82 views

What to do AFTER nested cross-validation?

I've searched exhaustively on this forum and elsewhere, and have come across a lot of great material. However, I'm ultimately still confused. Here's a basic, concrete example of what I'd like to ...
0
votes
0answers
29 views

ROC curve and its function beginner

I have 3 features of a signal (example: amplitude, frequency, energy). I want to check which feature is the best to represent that particular signal. That signal is classified into two categories ...
0
votes
0answers
13 views

SVD Down to One Dimension - K=1

I ran an analysis on a very sparse 40K x 40K customer-item rating matrix for recommendations; I first ran SVD on this matrix using many different reduced rank sizes, k=20,30,40... I used the results ...
0
votes
0answers
32 views

How do the ROC cutoffs relate to predictors?

Apologies for this rather simple question, but I haven't been able to find a definition online. What does the ROC cutoffs represent for the AUC package? Specifically, how does it relate to the ...
2
votes
3answers
128 views

The value of adding the ROC graph if the AUC is given

I always see in papers that when they want to show how they classifiers performed, they provide ROC graph and the AUC score. Now as far as I know only the AUC tells how well the classifier performed, ...
2
votes
2answers
112 views

Area under the ROC curve or area under the PR curve for imbalanced data?

I have some doubts about which performance measure to use, area under the ROC curve (TPR as a function of FPR) or area under the precision-recall curve (precision as a function of recall). My data is ...
5
votes
3answers
331 views

Why is AUC higher for a classifier that is less accurate than for one that is more accurate?

I have two classifiers A: naive Bayesian network B: tree (singly-connected) Bayesian network In terms of accuracy and other measures, A performs comparatively worse than B. However, when I use the ...
0
votes
0answers
7 views

help with AUC for PR curve when data has tied and skewed values

I am wondering if there are methods available to calculate AUC for Precision Recall curves when the predicted scores/probs/beliefs(whatever you want to call it) has tied values and could be skewed ...
3
votes
1answer
144 views

Statistics for Area under the ROC curve

I have a question regarding statistical evaluation of the AUC. In their paper (http://www.jstor.org/stable/2531595), DeLong et al. describe a method to evaluate AUC curves. (Another good explanation ...
0
votes
1answer
133 views

Predicting class probabilities in regression based on area under the curve

Logistic regression models the log odds. That is for rv $Y$ which is binary logit$(Y=1)=X\beta$. Then with this model, you can estimate the class probabilities and hence prediction or ...
2
votes
1answer
108 views

Comparison of two logistic regression models (significant result with anova() but very similar AUCs)

I have compared two logistic regression models using the function anova(mod1,mod2,test="Chisq") in R. The result that I obtained is the following: ...
1
vote
0answers
36 views

Calculate AUC of a logistic regression model [duplicate]

I have a data sample of a bank loan history of customers. I have performed logistic regression testing on the sample for finding out how the loan repayment(YES/NO) is dependent on various factors. I ...
1
vote
1answer
48 views

which performance metrics to classify model

I wonder between two performance metrics for classification models: accuracy and area under ROC curve (AUC), which one is to be preferred in which conditions? examples appreciated
0
votes
1answer
92 views

Accuracy and area under ROC curve (AUC)

If we group examples with and without class labels using clustering techniques by treating the class as an ordinary nominal attribute, the resulting clusters can then be used for classifying test ...
1
vote
1answer
156 views

Can someone sort me out regarding the calculation of AUC?

I am having some trouble with two different implementations of a classification problem giving different results. Me and my college who did the other implementation has narrowed the problem down to ...
0
votes
0answers
75 views

AUC is equivalent to a Mann-Whitney U-score, is the basic multiclass AUC related to the Kruskal-Wallis test statistic?

I've read that the area under the ROC curve is equivalent to a Mann-Whitney U-score. Is a multiclass AUC score (which averages the AUC scores for pairs of classes) related to the Kruskal-Wallis test ...
0
votes
0answers
20 views

Sample size when comparing several AUCs

How many individuals do I need to compare 3 AUC which are computed from the same set of patients? Is there a R program or some other program available? There are many programs for computing the ...
3
votes
3answers
194 views

pattern of ROC curve and choice of AUC

I am using ROC curves and full AUC values to compare different models, using simulated data. Now I think I am confused with the interpretations of ROC curves and AUC values. Please see the figure ...
3
votes
2answers
128 views

Estimating ROC/AUC on large data sets?

Plotting an ROC curve of a classifier compared to cases requires that the data set be sorted first on the classifier score. I am in a position where I need to calculate ROC on a large data set very ...
0
votes
1answer
208 views

ROC curves and AUC in simulations to compare models

I am using ROC curves to compare different methods but not sure if I need to re-simulate datasets using different seeds in R in order to reduce the "by-chance" issue for a particular output. Here is a ...
2
votes
3answers
383 views

Choosing a classification performance metric for model selection, feature selection, and publication

I have a small, unbalanced data set (70 positive, 30 negative), and I have been playing around with model selection for SVM parameters using BAC (balanced accuracy) and AUC (area under the curve). I ...
0
votes
1answer
82 views

Test to rank methods by AUCs on various benchmarks

Suppose I have N methods and M benchmarks. I have an AUC statistic (and some other similar statistics) for each combination of method with benchmark. What test should I use to test if one method is ...
1
vote
1answer
158 views

Different score range when calculating area of under curve in ROC curves

I have two classifiers which try to classify the same data sets. In order to check the efficiency of the classifiers I intend to plot the curves and calculate the AUC value. The concern is that one of ...
0
votes
0answers
143 views

How to use pROC package in R

I have a pool of factor numbers and a classifier which determines if a factor plays role in a disease or not. So the test result is "yes" or "no" which shows whether the factor involves in the disease ...
1
vote
1answer
195 views

Calculate LOO-AUC values using glmnet

I have a matrix (x) containing 55 samples (rows) and 10000 independent variables (columns). The observations are binary, healthy or ill {0,1} (y). I want to perform leave one out cross-validation and ...
1
vote
1answer
340 views

Calculation of AUC value from ROC Curve

Is there any tool that can calculate the AUC value from a ROC curve if I already know how many samples are true positive, true negative, false positive, false negative out of 500 samples? Specificity ...
0
votes
0answers
47 views

Tool for calculating AUC Value [duplicate]

Is there any tool that can calculate the AUC value from a ROC curve if I already have true positive, true negative, false positive, false negative values.
1
vote
1answer
211 views

How can I validate a logistic regression model using averaged parameter estimates?

Let me say thanks in advance. I'm working with a set of data that contains reported coyote sightings. I use 2/3 of the data for model calibration along with an equal number of pseudo absences. I ...
2
votes
3answers
179 views

How can I tell if my binary classifier is any good?

Say I have a data set with 10,000 rows and the target is a binary variable with 1500 positives (1's) and 8500 negatives (0's). I run a model and get predictions on the 0-1 interval. My question is ...
2
votes
2answers
133 views

Alternating AUC curve. What does it mean?

Why do I see my ROC curve crossing the line from (0,0) to (1,1) (i.e. the 0-1 line)? I have the following test data as a tab-separated testdata.txt file. Running my R code (given below) multiple ...
2
votes
1answer
142 views

AUC for more than two groups?

Standard ROC curves look at how setting various thresholds on a continuous measure can be used to predict a two-level ordinal outcome (example: antibody level -> (not sick, sick) ). This can then be ...
2
votes
0answers
118 views

Role of coefficients in model selection for logistic regression

I have a model that I am using to predict mortality and it gives me an AUC of 0.799. The R code that I am using would look something like this: ...
1
vote
0answers
96 views

Using AUC to compare logistic lasso and elastic net

I've seen this question answered here but I do not understand the answer. Harrell recommends using deviance based measures. David Hand (referenced in the thread) says that that the AUC is ...
4
votes
2answers
270 views

Compare classifiers based on AUROC or accuracy?

I have a binary classification problem and I experiment different classifiers on it: I want to compare the classifiers. which one is a better measure AUC or accuracy? And why? ...
9
votes
2answers
5k views

What is the difference in what AIC and c-statistic (AUC) actually measure for model fit?

Akaike Information Criterion (AIC) and the c-statistic (area under ROC curve) are two measures of model fit for logistic regression. I am having trouble explaining what is going on when the results of ...
1
vote
2answers
170 views

Is it possible to get high AUC while the correlation between predictor and response is very low, around 0.01?

My data has three continuous predictors and one binary response. I built a logistic regression model but AUC is only 0.52..it's almost like the model did nothing.. Then I calculated the correlation ...
1
vote
0answers
94 views

Intuition behind medium true positive rate, low false positive rate and “acceptable” AUC

I'm looking into some classification tasks at the moment. The test data is unbalanced where one particular class is half the data and the remaining 5 take up the remainder of the test data. When I ...
1
vote
2answers
135 views

How to generate ROC Plot for semi-supervised algorithm?

By having a data-set 1000 (900 unlabeled, 100 labeled) record data-set for binary classification, I want to apply a semi supervised algorithm. The problem is that I don't know how to get values for ...
2
votes
2answers
547 views

Fastest way to compare ROC curves

I have a set of true positive (TP) values which are used to train a model. I am using 5-fold cross validation to train my model (i.e. split my true positives into 5, use 4/5ths for training and ...
2
votes
1answer
417 views

Decision threshold for a 3-class Naive Bayes ROC curve

I have some doubts regarding how a ROC curve for a 3-class classifier (Naive Bayes) can be built. Basically, given some test data, the classifier outputs the probabilities for each of the 3 possible ...
2
votes
2answers
452 views

Recall and AUC of a binary classifier

Is it possible for a binary classifier to have a recall of 0.0 for one of the classes and at the same time an area under the ROC curve (AUC) of ...
5
votes
2answers
3k views

ROC curve for discrete classifiers like SVM: Why do we still call it a “curve”?, Isn't it just a “point”?

In the discussion : how to generate a roc curve for binary classification, I think that the confusion was that a "binary classifier" (which is any classifier that separates 2 classes) was for Yang ...
5
votes
1answer
347 views

Wilcoxon test in boot() function

I'm trying out the boot() function for internal validation of a logistic glm model using the AUC (aka c-statistic) as my performance measure. My problem is that depending on the dataset I use, ...