# Questions tagged [auc]

AUC stands for the Area Under the Curve and usually refers to the area under the receiver operator characteristic (ROC) curve.

473 questions
Filter by
Sorted by
Tagged with
7 views

### Comparing AUC of a binomial model to Cox for a specific time point

I have a model for 2 year mortality which was derived using binomial regression. Now I want to create a time-to-event model with Cox and I want to compare the performance to the already available ...
5 views

### Perfect recall but moderate precision due to imbalance?

I have a patient dataset on which I trained a RF classifier to predict whether a patient ends up in the hospital or not. Nevertheless, this dependent variable is imbalanced (66% of the patients ended ...
21 views

### When would AUC fail in comparing models? [closed]

It is possible that a classifier might have 1 threshold where there is highest possible true positive rate and least possible false positive rate (and lets say that is what the application requires), ...
7 views

### Convert RMSE, MSE, AUC, AUC PR or log loss to accuracy

I'm using h2o automl system and it returns all the models evaluation scores in MSE, RMSE, area under curve(AUC), AUC PR (dunno what this is) and log loss. I need this in accuracy so I can compare it ...
50 views

### Showing that $P(X_1>X_2) = \int_{0}^1 P(X_1>X_2 | X_2=x) f_{X_2}(x) dx$

I am going through this post in trying to prove the probabilistic interpretation of the AUC for a ROC Curve (for a classifier): The AUC for a ROC curve is the the probability of the classifier ...
13 views

### P-value for precision recall curve significance

I am computing the precision-recall curve for my ML model with 2 classes. I want to have a p-value that compare the observed area under the precision recall curve (AUPR_obs) and the area of such a ...
18 views

### Sklearn Average_Precision_Score vs. AUC

Can someone explain in an intuitive way the difference between Average_Precision_Score and AUC? I read the documentation and understand that they are calculated slightly differently. But what is the ...
8 views

### Is KS statstic (in classification) from ROC Curve same as K-S from cum Gain curve or PR Curve?

This is related to classification problems (specifically binary classification problems covered in scoring) where curves referenced below are used. I understand each of these curves for a classifier ...
23 views

### How is the cost weight function $w_G(c)$ implicitly used by the AUC calculated?

In this article by David Hand an implicit function of the classification cost ratio is calculated for a specific dataset, resulting in a discrete distribution: This is defined as  w_G(c) = \pi_0 ...
19 views

### Why did KNN Perform poorly with Error Rate but good with AUC

I have a set of error rates and AUC values for classification methods, Logistic Regression, QDA, LDA, and KNN. It looks like: ...
15 views

### Is the PR AUC invariant under label flip?

The ROC-AUC curve is invariant under a flip of the labels. I don't know if it's a famous result, so I will give the proof below. My question is if the PR-AUC curve also has this property. I have not ...
40 views

### Interpretation of AUC - ROC curves with a Binary Predictor

I have data like this: ...
7 views

### What is the right way to take a PR AUC average of a set of binary classifiers?

I have a multilabel classification problem. I am choosing to treat it as a set of independent binary classifiers. Each label has its corresponding skewness which can range from 1% to 20%. Suppose for ...
9 views

### How to evaluate differences in standard accuracy (PCC), Kappa, and AUC scores between random forest models?

I am using two sets of variables (life-history traits and human factors) to predict extinction Risk (Threatened/Non-Threatened) in turtles. In my first model (RF1 below) I only include life-history ...
38 views

### Precision-Recall Curve and Area under Precision-Recall Curve (AUC)

I created model (logistic regression) and now trying to create Precision-Recall plot and calculate area under Precision-Recall Plot. I'd like to note that this model is defective: ...
19 views

### Concordance estimate survival data in long format

I would like to use concordance to have an idea of my models discrimination power. Because I have time dependent covariates, the time to event data is in a "long" format, i.e. one row per ...
8 views

### AUC plot from a MLSeq::classify object

I have generated a classify object using the MLSeq::classify function. I wonder how I can visualise this using a ROC or AUC curve with sensitivity and specificity on the axis. ...
24 views

### Will the AUC of the full logistic regression model be equal to or higher than the AUC of models with less features?

I want a model to predict the probability of an event occurring. I would like to use logistic regression for this. An external condition that needs to be satisfied, however, to allow me to use such a ...
18 views

### Comparing AUCs: Same Control, different Test Group - paired or unpaired? pROC

I want to compare two AUCs using a bootstraping method from pROC package (roc.test). I have done a classification using Logistic Regression: 1. Cancer (high risk) vs. Control = AUC 1 2. Cancer (low ...
149 views

### How to generate a ROC curve from ground truth and predictions (in R package pROC)?

I've trained several machine learning models (GLM, SVM, random forest) in R to perform binary classification (predicting the presence of gallstones) and plotted ROC curves/computed AUC using the pROC ...
11 views

### What should I do when overfitting appears on AUC and not AUPR

I am training a classifier with imbalanced classes (pos/neg = 0.0006). When training the algorithm (xgboost in this case) I noticed that it shows an overfitting when we look at the AUC but the other ...
18 views

### AUC for crossvalidation

I have a medical research scenario where I am trying to predict disease progression. I need to produce a model to integrate into clinical decision support (and evaluate further). In addition to ...
31 views

### ROC Curve for unbounded scores

Say I have a classifier that assigns a score to an image based on whether it has a cat in it. The higher the score, the more likely there's a cat in it. But for this classifier, the value of the score ...
23 views

### How to decide which features are important in this binary classification task?

Consider a binary classification problem, where the dataset is highly imbalanced, with only around 20% positive labels and 80% negative labels. Feature A has higher AuROC when considering all the data,...
24 views

### ROC and AUC for clustering algorithms [closed]

I am working on some clustering algorithms like DBSCAN and local outlier factor. Now i want to know how can I make ROC and AUC curves from clustering results. Do anyone know how can i make RO and AUC ...
17 views

### Binary probability scoring: Intuition on why a method might perform better in terms of Brier, log loss but worse in terms of Area under ROC/PR curve?

I'm trying to compare two methods. I have surface knowledge about these scorers, so I've noticed that scorers in which method A performs better are both proper scoring rule, while B performs better in ...
17 views

### Understanding the DeLong vs t-test for comparing AUROCs

DeLong's paper was recommended to me for a statistical test to compare cross-validation AUCs $c_1$ and $c_2$ from two different classifiers on the same dataset. My understanding is that it is used to ...
23 views

### How to select best models if the ROC AUC score changes drastically at each separate run?

Below are two plots for ROC curves with their AUC mentioned in the legend brackets. How do I shortlist the best models if the scores differ at each run? Should I rather calculate the ROC AUC only from ...
33 views

### How can an ROC AUC and its 95%CI be estimated from repeated crossvalidation?

I'm using repeated 100 times 10-fold cross-validation to provide internal validation of a logistic regression model. What is the best way to obtain the ROC AUC and it's 95% CI from this data. If I ...
20 views

### Can AUC increase while both sensitivity and specificity decrease?

Assume I am given two classifiers - A and B. Is it possible that the specificity and sensitivity obtained from using B will be higher than A, but the AUC of A will be higher than the AUC of B? Will ...
28 views

### ROC Curves for Regression Output

I am working on a broad machine learning-based problem, which can be approached in several different ways. Essentially, my training values are floats between 0.0 and 1.0, and I have approached this in ...
49 views

### Is it correct to evaluate individual drivers with the AUC value?

I have a discussion with my supervisor about the usage of AUC to determine, basically, the importance of three different drivers consisting of multiple variables each. He claims I can look into the ...
22 views

### Feature selection for optimising LOOCV AUC

at the moment I am dealing with the following problem: I have a binary classification problem with low sample size (12 and 36 instances in the two classes) and large number of features (200). My ...
1k views

### Determine how good an AUC is (Area under the Curve of ROC)

I'm currently working on a project involving using different sets of data as a predictor to predict the outcome of out-sample data. I use AUC (Area under the Curve of ROC) to compare the performances ...
59 views

### R Studio: Calculate area under the curve with respect to ground

I would like to calculate an area under the curve with respect to ground (AUCg) according to this formula in this paper (Pruessner, Kirschbaum, Meinlschmid & Hellhammer, 2003). I have three ...
166 views

### Building AUC/ROC curve without probabilities, only with actual/predicted labels

If we dont have access to model and have just actual and predicted labels without probabilities, is it still be possible to plot AUC/ROC curve. For example can we have the curve from the following ...
38 views

### Can I draw the ROC curve from ROC_AUC Score?

The question may look rediculous. The problem in my hand is a multi-class (5 class labels) classification problem and I coded it a year back. I am having the resulsts such as G-mean, F-score, and ...
101 views

### Is the pooled AUC calculation for imputated data in (psfmi package) mivalext_lr() correct?

I have an imputated data with several nonmissing and not-imputated variables. However, I realised when I use mivalext_lr() to obtain pooled AUC and 95% CI of my ...
15 views

### All of my AU-PRC values are the same, is there something wrong with my code or models?

I have been doing some training of basic models for a certain binary outcome, and most of the training has been on optimizing the AUC. But when I plot the precision recall curves, I get essentially ...
25 views

### One-sided test for AUC (area under the curve for ROC)

I am trying to determine the power of testing the AUC of a diagnostic test with continuous score against a one-sided alternative. I was thinking I could just use the confidence intervals produced by <...
13 views

### Validation of flexible parametric survival models

Is there any dedicated R packages available for estimating AUC for flexible parametric survival models ("stpm2")?
22 views

### Generate synthetic data given AUC for multivariate X

I need to simulate data for a fixed AUC and for multivariate X. I came across the link below which explains for the univariate x. Generate synthetic data given AUC. Has anyone had any ideas or ...
47 views

### How can I rank preferences without ranking records in a ML model?

In a binary classification problem about the purchase of a product I use AUC to evaluate the performance of the model. Due to some restrictions I can't assign to each record of my data set any metric ...
13 views

### One AUC for each folder or Average AUC of all folders in Cross Validation?

Suppose I am running a leave-one-folder-out cross-validation (binary). The sample size of each folder is unbalanced. E.g., folder 1: X1 = [x11, x12, x13], y1 = [0,1,1] folder 2: X2 = [x21, x22], y2 = ...
22 views

### How to calculate the confidence interval of an AUC from its p-value?

Is there a way to calculate the confidence interval of an area under the curve (AUC) from its p-value? Let's say a study reports an AUC of 0.78 and a p-value of p = 0.001. Is there a way to calculate ...
34 views

### What's the connection between Kendall tau and AUC ROC? [duplicate]

What is the relationship between Kendall tau and the area under the receiver operating characteristic curve (AUC ROC)?
43 views

### Improve classification performance with H2O AutoML

I have to solve a simple binary classification problem using H2O AutoML. I would like yo optimize a custom loss function since the classification problem is inside a clear business framework and the ...
83 views

### Is the AUC an incoherent measure of classifier performance?

I'm learning about performance measures for binary classifiers. Reading about the AUC-ROC score I came across the article Measuring classifier performance: a coherent alternative to the area under the ...