an acronym for Area Under Curve.

learn more… | top users | synonyms

2
votes
1answer
33 views

Area under Precision-Recall Curve (AUC of PR-curve) and Average Precision (AP)

Is Average Precision (AP) the Area under Precision-Recall Curve (AUC of PR-curve) ? EDIT: here is some comment about difference in PR AUC and AP. The AUC is obtained by trapezoidal ...
1
vote
1answer
46 views

Area under ROC curve for random forest

Does the area under ROC curve depends on which class is defined as default positive class by the random forest model? I am using ...
0
votes
0answers
16 views

Measure for comparing serial measurements of multiple variables in treatment vs control group?

Can anyone suggest whether I am going about the statistics in an appropriate way? I have two groups of populations which have similar characteristics. In each member of both groups, serial ...
0
votes
1answer
32 views

Calculating AUC for a GEE

I have used the geeglm package to build a GEE that predicts animal activity (a binary response, active or not) from weather data (e.g., Temperature, a continuous variable). TEMPC <- ...
0
votes
0answers
10 views

Is there any lower limit for number of positives when generating lift plot?

I am wondering if there is any condition on number of positives in test set when I am trying to compute lift plot to check the properties of my classifier?
0
votes
1answer
73 views

How to calculate AUC for any correlation method?

I want to know how to calculate AUC to compare correlation methods. I read this paper http://www.ncbi.nlm.nih.gov/pubmed/23962479 Is there any idea how the authors of above paper have calculated AUC ...
1
vote
1answer
37 views

AUC per time is nothing else than the mean?

Using AUC is in vogue and has found his place also in clinical research (example). What I don't understand is AUC per time. For example, if a clinical or psychological parameter is measured over time. ...
0
votes
0answers
15 views

What are valid ways of analysing predictors for a response variable that changes with time?

I have a cohort of similar patients who are likely to get a certain disease over time. I am trying to find out how some continuous health markers (e.g. weight) at time 0 are related to their disease ...
2
votes
1answer
44 views

So many significant explanatory variables and so small auc

Have you ever seen a model with almost every significant variable and such small auc (area under the ROC curve) ? What might be the cause of it? When I saw summary of a model I thought this model will ...
0
votes
1answer
71 views

Random Forest - What training set measure is the best predictor of test set accuracy?

I'm running a random forest model on a training sample in R in order to make predictions on a hidden test set. I'm having difficulty in understanding how I should go about improving my model in order ...
1
vote
0answers
38 views

Comparing predictors based on ROC AUC and cross-validation error

I am analysing how well some continuous variables (e.g. weight, height) predict the occurrence of a given disease after surgery. I have computed the area under the curve of the receiver-operator ...
0
votes
0answers
37 views

How is the mean area under the curve calculated?

I am using 10-fold cross-validation for performance estimation. From each of the ten iterations, I get an area under the curve (AUC) metric, e.g. ...
3
votes
0answers
33 views

Optimizing for AUC

AUC is a popular classification evaluation metric. This is a measure of aggregate performance—do any of the standard loss functions (functions of an individual example's label & prediction) ...
0
votes
0answers
24 views

Forward sequential feature selection improving classifier performance?

I was in a bit of a conversation with a co-worker about using forward selection. My training data is on order of ~6,000 w/ dimensionality of 1,200, and testing data on order of ~3,000. Currently, I'm ...
1
vote
1answer
109 views

Equivalent of AUC (area under the ROC curve) for two variables

I was wondering if there is a way to compute AUC using two variables instead of one as predictors. I got two populations after a follow-up, divided in Cases and Controls according to whether they had ...
1
vote
0answers
114 views

How can I calculate AUC using Gini coefficient?

In the Gini Coefficient's Wikipedia page, it is defined as $G= 1 - \frac{\Sigma_{i=1}^n f(y_i)(S_{i-1}+S_i)}{S_n}$ for discrete variables, where $S_i= \Sigma_{j=1}^i f(y_i)y_i$ and $S_0=0$ ($y$ being ...
1
vote
0answers
122 views

Calculating two-tailed p-value from z-score for ROC AUC comparison

I am comparing two predictive models by their bootstrapped ROC AUCs with the method originally described by Hanley and McNeil and modified for bootstrapped data by Robin et al. I'm calculating the ...
0
votes
1answer
188 views

Training AUC and CV AUC in Boosted Regression Tree

My question is regarding the differences in the training data AUC score and the cross validation AUC score in boosted regression trees (BRT) built using the gbm.step function in the dismo package. I ...
1
vote
0answers
131 views
0
votes
1answer
69 views

Is there AUC for neural network?

I am confused about how to calculate AUC for neural network with a softmax classifier. For example, I know that for SVM, we can change the threshold value and determine the AUC. WHat about in neural ...
1
vote
0answers
28 views

measure the quality of feature vectors in terms of AUC and training time

I have computed 10 features for each example. I did an experiment to figure out which combinations of features can give the best classifier performance. I'm using libSVM for training. I noticed if I ...
20
votes
4answers
3k views

What does AUC stand for and what is it?

Searched high and low and have not been able to find out what AUC, as in related to prediction, stands for or means.
2
votes
1answer
338 views

Is it possible to get confusion matrices from AUC?

When I have one confusion matrix for each cutoff level (from 0.00 to 0.99), I can compute AUC coefficient. It looks like: ...
0
votes
0answers
158 views

How to evaluate the optimal cutoff of ROC curve related to logistic regression using roc from the R package pROC?

I would like to get the optimal cutoff of an ROC curve relating to a logistic regression. I am using the roc from the R package pROC. I am assuming same cost of false negative and false positive using ...
4
votes
0answers
125 views

Connections between d' (d-prime) and AUC (Area Under the ROC Curve); underlying assumptions

In machine learning we may use the area under the ROC curve (often abbreviated AUC, or AUROC) to summarise how well a system can discriminate between two categories. In signal detection theory often ...
0
votes
1answer
85 views

ROC / AUC for polynominal Labels

How can I calculate the Area Under Curve for a classifier of a plynominal label in Rapidminer? I could only find a performance operator for binominal labels that provides the AUC value.
0
votes
1answer
205 views

R - glmnet - cross validated - AUC [closed]

I have just started working with the glmnet package in R. I have s a dataset which has about 130,000 features and about 32000 rows of data. Here is the code to create the model ...
1
vote
1answer
89 views

What is AUC of PR-curve?

I understand that AUC under ROC curve is a classic evaluation measurement for classifiers (which is basically the accuracy). However, when data is imbalanced, PR will be alternative. So, what does the ...
0
votes
2answers
249 views

What are the differences between AUC and F1-score?

F1-score is the harmonic mean of precision and recall. The y-axis of recall is true positive rate (which is also recall). So, sometime classifiers can have low recall but very high AUC, what that ...
1
vote
1answer
247 views

Significant p value for Mann-Whitney U test but low AUC

How is it possible that for two sample sets I'm getting a low p-value, but also a low AUC value (just below 0.5)? To compute the P-value I'm looking at the second outputted value of the function here ...
0
votes
0answers
35 views

Cross Validation and perfcurv in Matlab

I am trying to use perfcurv in a cross validation code. However at some point all the members of the test dataset are of the same class (0). My problem is a binary classification problem. Therefore ...
1
vote
2answers
224 views

what does it mean when out of sample AUC is greater than in sample AUC?

I am fitting a logistic regression model on a data set with about 200,000 observation and 100 features. According to SAS output, the model converged correctly with an in-sample AUC of 0.85. However, ...
1
vote
0answers
184 views

Differences in AUC calculation in R between pROC and AUC

I was comparing the performance of pROC and AUC libraries when performing auc() calculations on random data: ...
2
votes
1answer
553 views

What is a good AUC for a precision-recall curve?

Because I have a very imbalanced dataset (9% positive outcomes), I decided a precision-recall curve was more appropriate than an ROC curve. I obtained the analogous summary measure of area under the ...
0
votes
1answer
53 views

Binary input to ROC analysis

Im working on assessment of algorithm sensitivity and specificity. I've developed a simulation in order to detect true and false positives and negatives. My intersest is to know if my algorithm is ...
2
votes
1answer
298 views

Is it reasonable for a classifier to obtain a high AUC and a low MCC? Or the opposite?

Let's say I have 2 models: 1) High Matthew's correlation coefficient (MCC) score, low area under the curve (AUC) 2) Low MCC, high AUC When I say high and low, I mean relatively to the other model. ...
0
votes
0answers
165 views

Multi-class AUC in Matlab

I would like to compute the area under the ROC-courve (AUC) metric for a classifier with multiple classes. Do you know (reliable) functions for Matlab that implement methods for that, like e.g. in ...
2
votes
1answer
262 views

Differences in AUC calculation between pROC and ROCR

Does anyone know the difference in calculation between these two AUC packages? They get different results when I add in positives with predicted value of 0 (simulating a prob model where many outputs ...
0
votes
0answers
24 views

Is the AUC for dataset (A union B) between the AUC of dataset A and the AUC of dataset B?

Consider you have a binary classifier which you tested on dataset AB=A union B. Assume that the several Area Under the Curve metrics for the three datasets are: AUC(A), AUC(B), and AUC(AB). Without ...
0
votes
1answer
23 views

Is Area under curve a composite function

I have some data examples. If I split the data into three parts and the have some scores for each example of the three parts and then calculate individual AUCs for the three parts In the next case, I ...
1
vote
1answer
126 views

ROC/AUC Confidence Interval

For a single ROC curve (with relevant AUC score), how can you calculate the confidence interval? (The data used to generate this ROC/AUC is available) Given my relatively limited background in this ...
1
vote
1answer
209 views

How can I get cut-off point in multivariated ROC analysis

If I have 1 independent variable (continues) and 1 dependent variable (binary), I can conduct logistic regression and ROC analysis, and I can get a cut-off point of independent variable using ROC ...
1
vote
1answer
506 views

R AUC never less than 0.5?

I'm doing some work with random forests in R using the randomForest package, and I've run into something that seems odd to me. Even when the data is completely ...
2
votes
1answer
1k views

Sample size calculation for ROC/AUC analysis

As a background, I am not familiar with stats except on a basic level. I have been tasked with doing some analysis that is out of my comfort zone. I am trying to figure out how to compute necessary ...
2
votes
1answer
255 views

Statistical Power of ROC/AUC Test with non-IID Samples :: To how many IID Samples are my non-IID Samples Equivalent?

I've been assigned to solve the following problem as part of a serious, biological research project. I think I have a tentative solution, but I'm wondering whether the approach I've picked is the ...
3
votes
2answers
167 views

Can AUC decrease with additional variables?

I'm fitting a logistic regression model to predict probabilities from a set of variables. I'm comparing two such models, say M1 and ...
1
vote
1answer
86 views

Reverse AUC interpretation

Given a classifier (SVM) classifying in 2 'classes' (+1 or -1) for prediction purposes. It has an AUC score of 0.28, meaning its success rate is lower than just random predictions. If I just do the ...
1
vote
0answers
68 views

Comparing AUC vs accuracy

I understand this question has been asked many times however, i am unable to understand the answers well enough and apply to my situation. I have attached 2 screenshots of my model. There are 5 class ...
3
votes
2answers
381 views

What to do AFTER nested cross-validation?

I've searched exhaustively on this forum and elsewhere, and have come across a lot of great material. However, I'm ultimately still confused. Here's a basic, concrete example of what I'd like to ...
2
votes
3answers
189 views

The value of adding the ROC graph if the AUC is given

I always see in papers that when they want to show how they classifiers performed, they provide ROC graph and the AUC score. Now as far as I know only the AUC tells how well the classifier performed, ...