an acronym for Area Under Curve.

learn more… | top users | synonyms

0
votes
0answers
18 views

How to calculate multi class classification AUC with labels? [closed]

I am using pROC (in R) with the function multiclass.roc as pointed out at the thread How to plot ROC curves in multiclass classification? However, when I applied ...
0
votes
0answers
10 views

Can I compute ROC AUC of F-measure for multi class classification? [duplicate]

I know ROC AUC is computed for binary classification, as well as F-score. But for multi - class classification, is it possible to calculate ROC AUC or F-score?
20
votes
4answers
3k views

What is the name of this chart showing false and true positive rates and how is it generated?

The image below shows a continuous curve of false positive rates vs. true positive rates: However, what I don't immediately get is how these rates are being calculated. If a method is applied to a ...
1
vote
0answers
9 views

how is it possible for a model with maximum AUC to not also have minumum misclassification error?

I have an elastic net model of a binary outcome where the lambda for max AUC is different than the lambda for min misclassification error. Shouldn't they be highly (inversely) correlated
0
votes
0answers
35 views

Use AUC for model comparison but what is the optimal threshold for final prediction

We can compare the performance of different models using AUC ROC and pick the one with large AUC. Then, we still need to choose and use specific threshold to predict the label for the test data. I ...
0
votes
0answers
18 views

In the classification framework, is AUROC a performance measure or metric?

I guess, the title is self-explaining. I have seen both so far and was wondering if there is a correct term or whether it does not really matter.
4
votes
1answer
38 views

Why two interpretations of AUC(area under the ROC curver) Equivalent

I found there are two ways to understand what AUC stands for but I couldn't get why these two interpretations are equivalent mathematically. In the first interpretation, AUC is the area under the ...
17
votes
1answer
660 views

Did I just invent a Bayesian method for analysis of ROC curves?

Preamble This is a long post. If you're re-reading this, please note that I've revised the question portion, though the background material remains the same. Additionally, I believe that I've devised ...
0
votes
1answer
38 views

Is it legitimate to use sensitivity and specificity next to more proper performance measures to compare classifiers?

Clearly, Brier Score and AUROC are better performance measures to compare classifiers. However, besides that, I am interested in a let's call it more economic view. I could imagine a classifier being ...
0
votes
2answers
48 views

How to interpret the AUROC curve for mortgage denial/approval?

My binary variable is whether a mortgage application is denied(1) or approved (0). Let's say I have two classifiers. One with AUROC = 0.75 and the other with AUROC = 0.85. Is it correct to state for ...
0
votes
0answers
46 views

How can we calculate ROC AUC for classification algorithm such as random forest?

As at In classification with 2 - classes, can a higher accuracy leads to a lower ROC - AUC?, AdamO said that for random forest ROC AUC is not available, because there is no cut-off value for this ...
0
votes
1answer
22 views

In classification with 2 - classes, can a higher accuracy leads to a lower ROC - AUC?

If I have a dataset with 2 possible outputs: Positive and Negative. I have 2 classification algorithms, each leads to a different predicting results. Is it possible if the algorithm 1 returns a ...
1
vote
1answer
82 views

Do these Precision-Recall (PR) curves indicate good classification performances?

I have trained a classifier for 3 different classes, the test datasets of which are imbalanced, and then plotted the PR curves (below) to evaluate their accuracies. The plots contain the number of ...
1
vote
1answer
44 views

Should PR AUC be used in cases where there is less than 5 positives vs 10000+ negatives?

I understand that the PR-AUC provides a better accuracy estimate than the ROC-AUC in the case of highly skewed datasets. But if I have a test dataset with less than 5 positives and 10000+ negatives, ...
0
votes
0answers
19 views

Binary Models - Deviance Logloss MSE AUC R2 Misclassification - Is there a defined choice?

For Binary Classification / Logistic Regression Models, Is there a specific preference or standard of what metric to be used for comparison of 2 models, especially when the model types are different - ...
0
votes
0answers
18 views

WEKA / Binary classifiers: Why two AUCs?

If I use different classification algorithms in WEKA, one possible output is the ROC-AUC. Why do I get two AUC indicators, one for the positive instances and one for the negative instances (besides ...
0
votes
1answer
123 views

Scoring a classifier with ROC AUC

I'm confused about how scikit-learn's roc_auc_score is working. As I understand it, an ROC AUC score for a classifier is obtained as follows: Sample from the parameter space Fit the model Make ...
2
votes
0answers
40 views

How to derive a mathematical formula for AUC?

Why the area under the ROC curve is the probability that a classifier will rank a randomly chosen "positive" instance (from the retrieved predictions) higher than a randomly chosen "positive" one ...
2
votes
2answers
57 views

SVM - can I use the decision function for calculating AUC?

An SVM returns a real-valued prediction for each of the input data samples, which corresponds to its distance from the separating hyperplane. Platt's scaling is often used to output a "probability" ...
2
votes
0answers
165 views

leave-one-out cross-validation on random data results in 0 AUC for glm model

I am running repeat simulations of leave-one-out cross-validation on glmnet models of randomly generated data, and collecting the AUC on left-out predictions (vs the full set of random targets). The ...
0
votes
1answer
105 views

Area Under the ROC Curve, a simple question

I split my dataset into 2 parts: 75% of it is the training set, 25% of it is the test set. Then I estimated the logistic regression parameters in the training set and I compute the Area Under the ROC ...
3
votes
1answer
67 views

Different ROC value for different packages in R, which one is correct?

I noticed that computing ROC with caret package and PROC packege sometimes gives different results. Usually they are the same, but if the predictions are worse than chance, caret will flip them and ...
0
votes
0answers
62 views

Is the ROC curve and estimated AUC from SPSS parametric or nonparametric?

I am using SPSS to generate some ROC curves, AUC and p values. According to SPSS manual, the AUC can be computed parametrically or nonparametrically. However, I do not see any option for that. There ...
0
votes
0answers
35 views

Confidence intervals for AUROC for repeated cross validation

I'm building a risk model using logistic regression in Stata. We are using $h \times k$ cross validation and calculating AUROC as part of the model validation procedure. Using Stata I've managed to ...
0
votes
0answers
52 views

GBM Performance on different sampling techniques

I am working on a healthcare data set for breast cancer patients. This data set is class imbalances and the distribution of positive and negative classes is 80%/20%. In order to deal with the class ...
1
vote
0answers
44 views

Does it make sense to minimize AUC when using GBM with weights?

I am using gbm(R's caret packages - using train function) on a class imbalanced data set with weights. So, class-1 has a weight of 1 and class-0 has a weight of 10. I am using parameter tuning and ...
2
votes
0answers
60 views

How to calculate AUC in Adaboost testing phase in R?

I'm using ROCR package in R for calculating AUC and drawing ROC curve using ...
0
votes
1answer
530 views

How can I calculate the AUC of combined variables using SPSS

thank you for taking time out to read this. I have previously ran ROC curves to get the AUCs for single test variables but I do not know how to derive the AUC for combined variables (2 test ...
0
votes
0answers
26 views

Diagnostic accuracy test done using mada package. How can I compare the AUC of SROC curves?

I am performing a meta-analysis to compare the diagnostic accuracy of different modalities on the same population. I have constructed SROC curves and have calculated the AUC values in each case. Is ...
2
votes
1answer
47 views

What should the AUROC be on the test set when no positive example is present?

Assume we have a probabilistic, binary classifier. We compute the AUROC on a test set in which no positive example is present (i.e. the ground truth is always 0). What should the AUROC be?
4
votes
1answer
104 views

What's the reasoning behind presenting unvalidated AUC as a measure of model fit or performance?

Often one sees, particularly in the biomedical literature, papers that analyze the performance of a risk prediction model in terms of the AUC or the area under the ROC curve. If the AUC is suitably ...
3
votes
2answers
413 views

How to interpret 95% confidence interval for Area Under Curve of ROC?

(I am following this paper, from page 47 on ...
0
votes
1answer
91 views

Differences between cross validation and bootstrapping to estimate the standard error of the AUC of a given ROC curve

I know there's been some discussion on differences between CV and bootstrapping for estimating out-of-sample prediction error of a classifier. For example, in here (Differences between cross ...
1
vote
0answers
29 views

Relationship between Accuracy Ration and Area Under Curve in discrete model?

How can we establish connection between AR (Accuracy Ration) and AUC (Area Under Curve) when AR depends on order of data? I am talking about relation $AR = 2AUC - 1$. Here ...
0
votes
0answers
67 views

pearson correlation vs AUC of ROC

is it possible to get not significant correlation with p>0.05, but the area under ROC curve is excellent with score 0.99?
0
votes
1answer
98 views

testing equivalence for two independent AUC

First of all, sorry for the "silly" question. I have two AUC, the first one comes from a training set and the other one comes from a validation set. I am using the ...
1
vote
1answer
198 views

Using partial AUC as Caret metric for cross-validation?

I'm evaluating a grid of tuning parameters using Caret with metric="ROC" for cross-validation. Is there any simple way to use as metric the area under the curve for an specified interval of the ROC ...
0
votes
1answer
112 views

Three way splitting and difference between CV AUC and testing AUC

I have 2000 observations in a dataset with features and a binary-class outcome. I split the dataset into two sets for split sample validation. I use 80% to train the model and internal perform Cross ...
2
votes
1answer
452 views

Area under Precision-Recall Curve (AUC of PR-curve) and Average Precision (AP)

Is Average Precision (AP) the Area under Precision-Recall Curve (AUC of PR-curve) ? EDIT: here is some comment about difference in PR AUC and AP. The AUC is obtained by trapezoidal ...
1
vote
1answer
530 views

Area under ROC curve for random forest

Does the area under ROC curve depends on which class is defined as default positive class by the random forest model? I am using ...
0
votes
0answers
51 views

Measure for comparing serial measurements of multiple variables in treatment vs control group?

Can anyone suggest whether I am going about the statistics in an appropriate way? I have two groups of populations which have similar characteristics. In each member of both groups, serial ...
0
votes
1answer
76 views

Calculating AUC for a GEE

I have used the geeglm package to build a GEE that predicts animal activity (a binary response, active or not) from weather data (e.g., Temperature, a continuous variable). TEMPC <- ...
0
votes
0answers
10 views

Is there any lower limit for number of positives when generating lift plot?

I am wondering if there is any condition on number of positives in test set when I am trying to compute lift plot to check the properties of my classifier?
0
votes
1answer
100 views

How to calculate AUC for any correlation method?

I want to know how to calculate AUC to compare correlation methods. I read this paper http://www.ncbi.nlm.nih.gov/pubmed/23962479 Is there any idea how the authors of above paper have calculated AUC ...
1
vote
1answer
43 views

AUC per time is nothing else than the mean?

Using AUC is in vogue and has found his place also in clinical research (example). What I don't understand is AUC per time. For example, if a clinical or psychological parameter is measured over time. ...
0
votes
0answers
26 views

What are valid ways of analysing predictors for a response variable that changes with time?

I have a cohort of similar patients who are likely to get a certain disease over time. I am trying to find out how some continuous health markers (e.g. weight) at time 0 are related to their disease ...
2
votes
1answer
59 views

So many significant explanatory variables and so small auc

Have you ever seen a model with almost every significant variable and such small auc (area under the ROC curve) ? What might be the cause of it? When I saw summary of a model I thought this model will ...
1
vote
1answer
257 views

Random Forest - What training set measure is the best predictor of test set accuracy?

I'm running a random forest model on a training sample in R in order to make predictions on a hidden test set. I'm having difficulty in understanding how I should go about improving my model in order ...
1
vote
0answers
104 views

Comparing predictors based on ROC AUC and cross-validation error

I am analysing how well some continuous variables (e.g. weight, height) predict the occurrence of a given disease after surgery. I have computed the area under the curve of the receiver-operator ...
0
votes
0answers
45 views

How is the mean area under the curve calculated?

I am using 10-fold cross-validation for performance estimation. From each of the ten iterations, I get an area under the curve (AUC) metric, e.g. ...