AUC is an acronym for Area Under Curve.

learn more… | top users | synonyms

0
votes
0answers
20 views

ROC curve and its function beginner

I have 3 features of a signal (example: amplitude, frequency, energy). I want to check which feature is the best to represent that particular signal. That signal is classified into two categories ...
0
votes
0answers
10 views

SVD Down to One Dimension - K=1

I ran an analysis on a very sparse 40K x 40K customer-item rating matrix for recommendations; I first ran SVD on this matrix using many different reduced rank sizes, k=20,30,40... I used the results ...
0
votes
0answers
23 views

How do the ROC cutoffs relate to predictors?

Apologies for this rather simple question, but I haven't been able to find a definition online. What does the ROC cutoffs represent for the AUC package? Specifically, how does it relate to the ...
2
votes
3answers
97 views

The value of adding the ROC graph if the AUC is given

I always see in papers that when they want to show how they classifiers performed, they provide ROC graph and the AUC score. Now as far as I know only the AUC tells how well the classifier performed, ...
1
vote
2answers
52 views

Area under the ROC curve or area under the PR curve for imbalanced data?

I have some doubts about which performance measure to use, area under the ROC curve (TPR as a function of FPR) or area under the precision-recall curve (precision as a function of recall). My data is ...
5
votes
3answers
240 views

Why is AUC higher for a classifier that is less accurate than for one that is more accurate?

I have two classifiers A: naive Bayesian network B: tree (singly-connected) Bayesian network In terms of accuracy and other measures, A performs comparatively worse than B. However, when I use the ...
0
votes
0answers
3 views

help with AUC for PR curve when data has tied and skewed values

I am wondering if there are methods available to calculate AUC for Precision Recall curves when the predicted scores/probs/beliefs(whatever you want to call it) has tied values and could be skewed ...
2
votes
1answer
87 views

Statistics for Area under the ROC curve

I have a question regarding statistical evaluation of the AUC. In their paper (http://www.jstor.org/stable/2531595), DeLong et al. describe a method to evaluate AUC curves. (Another good explanation ...
0
votes
1answer
77 views

Predicting class probabilities in regression based on area under the curve

Logistic regression models the log odds. That is for rv $Y$ which is binary logit$(Y=1)=X\beta$. Then with this model, you can estimate the class probabilities and hence prediction or ...
2
votes
1answer
74 views

Comparison of two logistic regression models (significant result with anova() but very similar AUCs)

I have compared two logistic regression models using the function anova(mod1,mod2,test="Chisq") in R. The result that I obtained is the following: ...
1
vote
0answers
32 views

Calculate AUC of a logistic regression model [duplicate]

I have a data sample of a bank loan history of customers. I have performed logistic regression testing on the sample for finding out how the loan repayment(YES/NO) is dependent on various factors. I ...
1
vote
1answer
38 views

which performance metrics to classify model

I wonder between two performance metrics for classification models: accuracy and area under ROC curve (AUC), which one is to be preferred in which conditions? examples appreciated
0
votes
1answer
63 views

Accuracy and area under ROC curve (AUC)

If we group examples with and without class labels using clustering techniques by treating the class as an ordinary nominal attribute, the resulting clusters can then be used for classifying test ...
1
vote
1answer
113 views

Can someone sort me out regarding the calculation of AUC?

I am having some trouble with two different implementations of a classification problem giving different results. Me and my college who did the other implementation has narrowed the problem down to ...
0
votes
0answers
53 views

AUC is equivalent to a Mann-Whitney U-score, is the basic multiclass AUC related to the Kruskal-Wallis test statistic?

I've read that the area under the ROC curve is equivalent to a Mann-Whitney U-score. Is a multiclass AUC score (which averages the AUC scores for pairs of classes) related to the Kruskal-Wallis test ...
0
votes
0answers
18 views

Sample size when comparing several AUCs

How many individuals do I need to compare 3 AUC which are computed from the same set of patients? Is there a R program or some other program available? There are many programs for computing the ...
3
votes
3answers
148 views

pattern of ROC curve and choice of AUC

I am using ROC curves and full AUC values to compare different models, using simulated data. Now I think I am confused with the interpretations of ROC curves and AUC values. Please see the figure ...
3
votes
2answers
109 views

Estimating ROC/AUC on large data sets?

Plotting an ROC curve of a classifier compared to cases requires that the data set be sorted first on the classifier score. I am in a position where I need to calculate ROC on a large data set very ...
0
votes
1answer
141 views

ROC curves and AUC in simulations to compare models

I am using ROC curves to compare different methods but not sure if I need to re-simulate datasets using different seeds in R in order to reduce the "by-chance" issue for a particular output. Here is a ...
2
votes
3answers
268 views

Choosing a classification performance metric for model selection, feature selection, and publication

I have a small, unbalanced data set (70 positive, 30 negative), and I have been playing around with model selection for SVM parameters using BAC (balanced accuracy) and AUC (area under the curve). I ...
0
votes
1answer
54 views

Test to rank methods by AUCs on various benchmarks

Suppose I have N methods and M benchmarks. I have an AUC statistic (and some other similar statistics) for each combination of method with benchmark. What test should I use to test if one method is ...
1
vote
1answer
102 views

Different score range when calculating area of under curve in ROC curves

I have two classifiers which try to classify the same data sets. In order to check the efficiency of the classifiers I intend to plot the curves and calculate the AUC value. The concern is that one of ...
0
votes
0answers
129 views

How to use pROC package in R

I have a pool of factor numbers and a classifier which determines if a factor plays role in a disease or not. So the test result is "yes" or "no" which shows whether the factor involves in the disease ...
1
vote
1answer
136 views

Calculate LOO-AUC values using glmnet

I have a matrix (x) containing 55 samples (rows) and 10000 independent variables (columns). The observations are binary, healthy or ill {0,1} (y). I want to perform leave one out cross-validation and ...
1
vote
1answer
281 views

Calculation of AUC value from ROC Curve

Is there any tool that can calculate the AUC value from a ROC curve if I already know how many samples are true positive, true negative, false positive, false negative out of 500 samples? Specificity ...
0
votes
0answers
47 views

Tool for calculating AUC Value [duplicate]

Is there any tool that can calculate the AUC value from a ROC curve if I already have true positive, true negative, false positive, false negative values.
1
vote
1answer
141 views

How can I validate a logistic regression model using averaged parameter estimates?

Let me say thanks in advance. I'm working with a set of data that contains reported coyote sightings. I use 2/3 of the data for model calibration along with an equal number of pseudo absences. I ...
2
votes
3answers
156 views

How can I tell if my binary classifier is any good?

Say I have a data set with 10,000 rows and the target is a binary variable with 1500 positives (1's) and 8500 negatives (0's). I run a model and get predictions on the 0-1 interval. My question is ...
2
votes
2answers
121 views

Alternating AUC curve. What does it mean?

Why do I see my ROC curve crossing the line from (0,0) to (1,1) (i.e. the 0-1 line)? I have the following test data as a tab-separated testdata.txt file. Running my R code (given below) multiple ...
2
votes
1answer
130 views

AUC for more than two groups?

Standard ROC curves look at how setting various thresholds on a continuous measure can be used to predict a two-level ordinal outcome (example: antibody level -> (not sick, sick) ). This can then be ...
2
votes
0answers
114 views

Role of coefficients in model selection for logistic regression

I have a model that I am using to predict mortality and it gives me an AUC of 0.799. The R code that I am using would look something like this: ...
1
vote
0answers
82 views

Using AUC to compare logistic lasso and elastic net

I've seen this question answered here but I do not understand the answer. Harrell recommends using deviance based measures. David Hand (referenced in the thread) says that that the AUC is ...
4
votes
2answers
236 views

Compare classifiers based on AUROC or accuracy?

I have a binary classification problem and I experiment different classifiers on it: I want to compare the classifiers. which one is a better measure AUC or accuracy? And why? ...
9
votes
2answers
4k views

What is the difference in what AIC and c-statistic (AUC) actually measure for model fit?

Akaike Information Criterion (AIC) and the c-statistic (area under ROC curve) are two measures of model fit for logistic regression. I am having trouble explaining what is going on when the results of ...
1
vote
2answers
136 views

Is it possible to get high AUC while the correlation between predictor and response is very low, around 0.01?

My data has three continuous predictors and one binary response. I built a logistic regression model but AUC is only 0.52..it's almost like the model did nothing.. Then I calculated the correlation ...
1
vote
0answers
89 views

Intuition behind medium true positive rate, low false positive rate and “acceptable” AUC

I'm looking into some classification tasks at the moment. The test data is unbalanced where one particular class is half the data and the remaining 5 take up the remainder of the test data. When I ...
1
vote
2answers
121 views

How to generate ROC Plot for semi-supervised algorithm?

By having a data-set 1000 (900 unlabeled, 100 labeled) record data-set for binary classification, I want to apply a semi supervised algorithm. The problem is that I don't know how to get values for ...
2
votes
2answers
487 views

Fastest way to compare ROC curves

I have a set of true positive (TP) values which are used to train a model. I am using 5-fold cross validation to train my model (i.e. split my true positives into 5, use 4/5ths for training and ...
2
votes
1answer
354 views

Decision threshold for a 3-class Naive Bayes ROC curve

I have some doubts regarding how a ROC curve for a 3-class classifier (Naive Bayes) can be built. Basically, given some test data, the classifier outputs the probabilities for each of the 3 possible ...
2
votes
2answers
397 views

Recall and AUC of a binary classifier

Is it possible for a binary classifier to have a recall of 0.0 for one of the classes and at the same time an area under the ROC curve (AUC) of ...
3
votes
2answers
3k views

ROC curve for discrete classifiers like SVM: Why do we still call it a “curve”?, Isn't it just a “point”?

In the discussion : how to generate a roc curve for binary classification, I think that the confusion was that a "binary classifier" (which is any classifier that separates 2 classes) was for Yang ...
5
votes
1answer
305 views

Wilcoxon test in boot() function

I'm trying out the boot() function for internal validation of a logistic glm model using the AUC (aka c-statistic) as my performance measure. My problem is that depending on the dataset I use, ...
2
votes
1answer
186 views

AUC vs error rate for classification

I'm trying to build a recommendation system, and have a bunch of (item,item_features,liked) triplets, where liked is binary. Most items are not liked. So I'm running a logistic regression with ...
3
votes
1answer
1k views

Lorenz curve and Gini coefficient for measuring classifier performance

I often use a ROC curve and the area under that curve as a measure of classifier accuracy in 2-class problems, e.g: ...
6
votes
2answers
516 views

AUC in ordinal logistic regression

I'm using 2 kind of logistic regression - one is the simple type, for binary classification, and the other is ordinal logistic regression. For calculating the accuracy of the first, I used ...
2
votes
1answer
496 views

Confidence interval and statistical significance in comparison of AUC

Recently, I have compared two correlated AUC with the method of Delong. Someone said that since the CI’s overlap, we cannot state the two models were different. I know that the method of Delong ...
4
votes
2answers
2k views

Area under the “pdf” in kernel density estimation in R

I am trying to use the 'density' function in R to do kernel density estimates. I am having some difficulty interpreting the results and comparing various datasets as it seems the area under the curve ...
4
votes
1answer
148 views

Empirical AUC in validation set when no TRUE zeroes

In a cross-validation setting (LASSO penalized logistic regression), I'm calculating AUC. However, I'm interested in the variability of these estimates over the folds (this will give me an indication ...