an acronym for Area Under Curve.

learn more… | top users | synonyms

0
votes
0answers
10 views

Relationship between Accuracy Ration and Area Under Curve in discrete model?

How can we establish connection between AR (Accuracy Ration) and AUC (Area Under Curve) when AR depends on order of data? I am talking about relation $AR = 2AUC - 1$. Here ...
0
votes
0answers
16 views

pearson correlation vs AUC of ROC

is it possible to get not significant correlation with p>0.05, but the area under ROC curve is excellent with score 0.99?
0
votes
1answer
17 views

testing equivalence for two independent AUC

First of all, sorry for the "silly" question. I have two AUC, the first one comes from a training set and the other one comes from a validation set. I am using the ...
0
votes
1answer
34 views

Using partial AUC as Caret metric for cross-validation?

I'm evaluating a grid of tuning parameters using Caret with metric="ROC" for cross-validation. Is there any simple way to use as metric the area under the curve for an specified interval of the ROC ...
0
votes
1answer
21 views

Three way splitting and difference between CV AUC and testing AUC

I have 2000 observations in a dataset with features and a binary-class outcome. I split the dataset into two sets for split sample validation. I use 80% to train the model and internal perform Cross ...
2
votes
1answer
63 views

Area under Precision-Recall Curve (AUC of PR-curve) and Average Precision (AP)

Is Average Precision (AP) the Area under Precision-Recall Curve (AUC of PR-curve) ? EDIT: here is some comment about difference in PR AUC and AP. The AUC is obtained by trapezoidal ...
1
vote
1answer
91 views

Area under ROC curve for random forest

Does the area under ROC curve depends on which class is defined as default positive class by the random forest model? I am using ...
0
votes
0answers
19 views

Measure for comparing serial measurements of multiple variables in treatment vs control group?

Can anyone suggest whether I am going about the statistics in an appropriate way? I have two groups of populations which have similar characteristics. In each member of both groups, serial ...
0
votes
1answer
40 views

Calculating AUC for a GEE

I have used the geeglm package to build a GEE that predicts animal activity (a binary response, active or not) from weather data (e.g., Temperature, a continuous variable). TEMPC <- ...
0
votes
0answers
10 views

Is there any lower limit for number of positives when generating lift plot?

I am wondering if there is any condition on number of positives in test set when I am trying to compute lift plot to check the properties of my classifier?
0
votes
1answer
79 views

How to calculate AUC for any correlation method?

I want to know how to calculate AUC to compare correlation methods. I read this paper http://www.ncbi.nlm.nih.gov/pubmed/23962479 Is there any idea how the authors of above paper have calculated AUC ...
1
vote
1answer
38 views

AUC per time is nothing else than the mean?

Using AUC is in vogue and has found his place also in clinical research (example). What I don't understand is AUC per time. For example, if a clinical or psychological parameter is measured over time. ...
0
votes
0answers
17 views

What are valid ways of analysing predictors for a response variable that changes with time?

I have a cohort of similar patients who are likely to get a certain disease over time. I am trying to find out how some continuous health markers (e.g. weight) at time 0 are related to their disease ...
2
votes
1answer
45 views

So many significant explanatory variables and so small auc

Have you ever seen a model with almost every significant variable and such small auc (area under the ROC curve) ? What might be the cause of it? When I saw summary of a model I thought this model will ...
0
votes
1answer
92 views

Random Forest - What training set measure is the best predictor of test set accuracy?

I'm running a random forest model on a training sample in R in order to make predictions on a hidden test set. I'm having difficulty in understanding how I should go about improving my model in order ...
1
vote
0answers
48 views

Comparing predictors based on ROC AUC and cross-validation error

I am analysing how well some continuous variables (e.g. weight, height) predict the occurrence of a given disease after surgery. I have computed the area under the curve of the receiver-operator ...
0
votes
0answers
38 views

How is the mean area under the curve calculated?

I am using 10-fold cross-validation for performance estimation. From each of the ten iterations, I get an area under the curve (AUC) metric, e.g. ...
3
votes
0answers
36 views

Optimizing for AUC

AUC is a popular classification evaluation metric. This is a measure of aggregate performance—do any of the standard loss functions (functions of an individual example's label & prediction) ...
0
votes
0answers
28 views

Forward sequential feature selection improving classifier performance?

I was in a bit of a conversation with a co-worker about using forward selection. My training data is on order of ~6,000 w/ dimensionality of 1,200, and testing data on order of ~3,000. Currently, I'm ...
1
vote
1answer
129 views

Equivalent of AUC (area under the ROC curve) for two variables

I was wondering if there is a way to compute AUC using two variables instead of one as predictors. I got two populations after a follow-up, divided in Cases and Controls according to whether they had ...
1
vote
0answers
171 views

How can I calculate AUC using Gini coefficient?

In the Gini Coefficient's Wikipedia page, it is defined as $G= 1 - \frac{\Sigma_{i=1}^n f(y_i)(S_{i-1}+S_i)}{S_n}$ for discrete variables, where $S_i= \Sigma_{j=1}^i f(y_i)y_i$ and $S_0=0$ ($y$ being ...
1
vote
0answers
133 views

Calculating two-tailed p-value from z-score for ROC AUC comparison

I am comparing two predictive models by their bootstrapped ROC AUCs with the method originally described by Hanley and McNeil and modified for bootstrapped data by Robin et al. I'm calculating the ...
0
votes
1answer
204 views

Training AUC and CV AUC in Boosted Regression Tree

My question is regarding the differences in the training data AUC score and the cross validation AUC score in boosted regression trees (BRT) built using the gbm.step function in the dismo package. I ...
1
vote
0answers
146 views
0
votes
1answer
72 views

Is there AUC for neural network?

I am confused about how to calculate AUC for neural network with a softmax classifier. For example, I know that for SVM, we can change the threshold value and determine the AUC. WHat about in neural ...
1
vote
0answers
30 views

measure the quality of feature vectors in terms of AUC and training time

I have computed 10 features for each example. I did an experiment to figure out which combinations of features can give the best classifier performance. I'm using libSVM for training. I noticed if I ...
21
votes
4answers
4k views

What does AUC stand for and what is it?

Searched high and low and have not been able to find out what AUC, as in related to prediction, stands for or means.
2
votes
1answer
373 views

Is it possible to get confusion matrices from AUC?

When I have one confusion matrix for each cutoff level (from 0.00 to 0.99), I can compute AUC coefficient. It looks like: ...
0
votes
0answers
176 views

How to evaluate the optimal cutoff of ROC curve related to logistic regression using roc from the R package pROC?

I would like to get the optimal cutoff of an ROC curve relating to a logistic regression. I am using the roc from the R package pROC. I am assuming same cost of false negative and false positive using ...
4
votes
0answers
138 views

Connections between d' (d-prime) and AUC (Area Under the ROC Curve); underlying assumptions

In machine learning we may use the area under the ROC curve (often abbreviated AUC, or AUROC) to summarise how well a system can discriminate between two categories. In signal detection theory often ...
0
votes
1answer
101 views

ROC / AUC for polynominal Labels

How can I calculate the Area Under Curve for a classifier of a plynominal label in Rapidminer? I could only find a performance operator for binominal labels that provides the AUC value.
0
votes
1answer
235 views

R - glmnet - cross validated - AUC [closed]

I have just started working with the glmnet package in R. I have s a dataset which has about 130,000 features and about 32000 rows of data. Here is the code to create the model ...
1
vote
1answer
99 views

What is AUC of PR-curve?

I understand that AUC under ROC curve is a classic evaluation measurement for classifiers (which is basically the accuracy). However, when data is imbalanced, PR will be alternative. So, what does the ...
0
votes
2answers
292 views

What are the differences between AUC and F1-score?

F1-score is the harmonic mean of precision and recall. The y-axis of recall is true positive rate (which is also recall). So, sometime classifiers can have low recall but very high AUC, what that ...
1
vote
1answer
267 views

Significant p value for Mann-Whitney U test but low AUC

How is it possible that for two sample sets I'm getting a low p-value, but also a low AUC value (just below 0.5)? To compute the P-value I'm looking at the second outputted value of the function here ...
0
votes
0answers
35 views

Cross Validation and perfcurv in Matlab

I am trying to use perfcurv in a cross validation code. However at some point all the members of the test dataset are of the same class (0). My problem is a binary classification problem. Therefore ...
1
vote
2answers
258 views

what does it mean when out of sample AUC is greater than in sample AUC?

I am fitting a logistic regression model on a data set with about 200,000 observation and 100 features. According to SAS output, the model converged correctly with an in-sample AUC of 0.85. However, ...
1
vote
0answers
204 views

Differences in AUC calculation in R between pROC and AUC

I was comparing the performance of pROC and AUC libraries when performing auc() calculations on random data: ...
2
votes
1answer
617 views

What is a good AUC for a precision-recall curve?

Because I have a very imbalanced dataset (9% positive outcomes), I decided a precision-recall curve was more appropriate than an ROC curve. I obtained the analogous summary measure of area under the ...
0
votes
1answer
57 views

Binary input to ROC analysis

Im working on assessment of algorithm sensitivity and specificity. I've developed a simulation in order to detect true and false positives and negatives. My intersest is to know if my algorithm is ...
2
votes
1answer
338 views

Is it reasonable for a classifier to obtain a high AUC and a low MCC? Or the opposite?

Let's say I have 2 models: 1) High Matthew's correlation coefficient (MCC) score, low area under the curve (AUC) 2) Low MCC, high AUC When I say high and low, I mean relatively to the other model. ...
0
votes
0answers
177 views

Multi-class AUC in Matlab

I would like to compute the area under the ROC-courve (AUC) metric for a classifier with multiple classes. Do you know (reliable) functions for Matlab that implement methods for that, like e.g. in ...
2
votes
1answer
291 views

Differences in AUC calculation between pROC and ROCR

Does anyone know the difference in calculation between these two AUC packages? They get different results when I add in positives with predicted value of 0 (simulating a prob model where many outputs ...
0
votes
0answers
25 views

Is the AUC for dataset (A union B) between the AUC of dataset A and the AUC of dataset B?

Consider you have a binary classifier which you tested on dataset AB=A union B. Assume that the several Area Under the Curve metrics for the three datasets are: AUC(A), AUC(B), and AUC(AB). Without ...
0
votes
1answer
26 views

Is Area under curve a composite function

I have some data examples. If I split the data into three parts and the have some scores for each example of the three parts and then calculate individual AUCs for the three parts In the next case, I ...
1
vote
1answer
135 views

ROC/AUC Confidence Interval

For a single ROC curve (with relevant AUC score), how can you calculate the confidence interval? (The data used to generate this ROC/AUC is available) Given my relatively limited background in this ...
1
vote
1answer
234 views

How can I get cut-off point in multivariated ROC analysis

If I have 1 independent variable (continues) and 1 dependent variable (binary), I can conduct logistic regression and ROC analysis, and I can get a cut-off point of independent variable using ROC ...
1
vote
1answer
569 views

R AUC never less than 0.5?

I'm doing some work with random forests in R using the randomForest package, and I've run into something that seems odd to me. Even when the data is completely ...
2
votes
1answer
1k views

Sample size calculation for ROC/AUC analysis

As a background, I am not familiar with stats except on a basic level. I have been tasked with doing some analysis that is out of my comfort zone. I am trying to figure out how to compute necessary ...
2
votes
1answer
258 views

Statistical Power of ROC/AUC Test with non-IID Samples :: To how many IID Samples are my non-IID Samples Equivalent?

I've been assigned to solve the following problem as part of a serious, biological research project. I think I have a tentative solution, but I'm wondering whether the approach I've picked is the ...