# Questions tagged [auc]

AUC stands for the Area Under the Curve and usually refers to the area under the receiver operator characteristic (ROC) curve.

443 questions
Filter by
Sorted by
Tagged with
8 views

### R Studio: Calculate area under the curve with respect to ground

I would like to calculate an area under the curve with respect to ground (AUCg) according to this formula in this paper (Pruessner, Kirschbaum, Meinlschmid & Hellhammer, 2003). I have three ...
36 views

### Building AUC/ROC curve without probabilities, only with actual/predicted labels

If we dont have access to model and have just actual and predicted labels without probabilities, is it still be possible to plot AUC/ROC curve. For example can we have the curve from the following ...
34 views

### Can I draw the ROC curve from ROC_AUC Score?

The question may look rediculous. The problem in my hand is a multi-class (5 class labels) classification problem and I coded it a year back. I am having the resulsts such as G-mean, F-score, and ...
21 views

### Is the pooled AUC calculation for imputated data in (psfmi package) mivalext_lr() correct?

I have an imputated data with several nonmissing and not-imputated variables. However, I realised when I use mivalext_lr() to obtain pooled AUC and 95% CI of my ...
38 views

### confused AUC ROC score [migrated]

I am working on binary classification problem, I try to evaluate the performance of some classification algorithms (LR,Decission Tree , Random forest ...). I am using a 10 fold cross-validation ...
11 views

### All of my AU-PRC values are the same, is there something wrong with my code or models?

I have been doing some training of basic models for a certain binary outcome, and most of the training has been on optimizing the AUC. But when I plot the precision recall curves, I get essentially ...
13 views

### One-sided test for AUC (area under the curve for ROC)

I am trying to determine the power of testing the AUC of a diagnostic test with continuous score against a one-sided alternative. I was thinking I could just use the confidence intervals produced by <...
9 views

### Validation of flexible parametric survival models

Is there any dedicated R packages available for estimating AUC for flexible parametric survival models ("stpm2")?
22 views

### Generate synthetic data given AUC for multivariate X

I need to simulate data for a fixed AUC and for multivariate X. I came across the link below which explains for the univariate x. Generate synthetic data given AUC. Has anyone had any ideas or ...
45 views

### How can I rank preferences without ranking records in a ML model?

In a binary classification problem about the purchase of a product I use AUC to evaluate the performance of the model. Due to some restrictions I can't assign to each record of my data set any metric ...
13 views

### One AUC for each folder or Average AUC of all folders in Cross Validation?

Suppose I am running a leave-one-folder-out cross-validation (binary). The sample size of each folder is unbalanced. E.g., folder 1: X1 = [x11, x12, x13], y1 = [0,1,1] folder 2: X2 = [x21, x22], y2 = ...
19 views

### How to calculate the confidence interval of an AUC from its p-value?

Is there a way to calculate the confidence interval of an area under the curve (AUC) from its p-value? Let's say a study reports an AUC of 0.78 and a p-value of p = 0.001. Is there a way to calculate ...
13 views

### What's the connection between Kendall tau and AUC ROC? [duplicate]

What is the relationship between Kendall tau and the area under the receiver operating characteristic curve (AUC ROC)?
16 views

### Improve classification performance with H2O AutoML

I have to solve a simple binary classification problem using H2O AutoML. I would like yo optimize a custom loss function since the classification problem is inside a clear business framework and the ...
40 views

### Is the AUC an incoherent measure of classifier performance?

I'm learning about performance measures for binary classifiers. Reading about the AUC-ROC score I came across the article Measuring classifier performance: a coherent alternative to the area under the ...
98 views

### Is my AUC too good to be true?

I've been going in circles for months...I want to generate a list of thresholds from my training data so I can see all the thresholds at every sensitivity/1-specificity of a model. You can do this ...
64 views

### Does test AUC of 0.98 mean overfitting if we have highly imbalanced dataset (0.5% minority class)?

I built a Random Forest model to classify imbalanced data (0.5% for minority class.) I used grid search to hyper tune parameters. I got the test AUC of almost 0.99. The test AUC is completely out of ...
37 views

### Do repeated measures affect point estimates of predicted probabilities or AUC from logistic regression?

I understand the presence of repeated measures affects standard errors and hence inferences using p-values, confidence intervals, etc. However, do repeated measures affect the point estimates of ...
29 views

### Cross Validation Random Forest AUC very high

first time poster so hopefully this makes sense and please let me know of any important information I may leave out. I'm trying to identify my cross-validated AUCs for basic GLM, elastic net, and ...
47 views

### Logarithmic loss vs Brier score vs AUC score

I have a dataset with two classes of elements. I also have two methods which assign (complementary) probabilities to each element in the dataset of belonging to either class. Given that I work with ...
31 views

### H2o Python - How do I determine the threshold for AUC

I am new to H2o and I having trouble with AUC and Confusion Matrix I have a classification model using H2o in Python for which the AUC = 71% But the accuracy based on confusion Matrix is only 61%. I ...
24 views

### Models have high accuracy but very low AUC PR curve

I'm doing balanced (70%-30%) binary classification (yes/no) I'm trying to combine caret train objects with PRROC's pr.curve function. I'm using a confusion matrix to determine which class is labelled ...
42 views

### When is a dataset “too imbalanced” for AUC ROC and PR is preferred?

I’ve read that precision-recall (PR) curves are preferred over AUC-ROC curves when a dataset is imbalanced as there’s more of a focus on the model’s performance in correctly identifying the minority/...
14 views

### What is a good PR-AUC and should I undersample time series for rare event detection? [duplicate]

I have a binary classifier for a highly imbalanced multivariate time series. I use an LSTM Network to predict the next time step and use the prediction error to decide whether a data point is an ...
26 views

### Q: AUC of each subgroup is smaller than overall AUC

I have a validation data set of 29242 patients, with known labels/health outcomes and predictions that were generated by some model. 28626 patients are negative and 616 are positive The overall AUC is ...
14 views

### Can a classifier's ROC AUC be compared to a person's accuracy at correctly ranking a positive, negative pair?

I've read that ROC AUC is equivalent to the probability of correctly ranking a positive, negative pair. If a human were to attempt to correctly rank a bunch of positive, negative pairs, could their ...
43 views

### Understanding AUC curve [closed]

This post is hidden. You deleted this post 12 hours ago. enter image description here In this image, i have a series of threshold as follows 0.5, 0.8, 0.85, 0.95. now in this diagram, if i choose the ...
7 views

### How to draw Receiver Operating Characteristics (ROC) of 34 video sequences using MatLab

I am working on video processing using MatLab R2017a. I have datasets containing 34 video sequences. I have to calculate the AUC of this datasets. As I know AUC is dependent upon the ROC. And ROC is a ...
23 views

### How can logistic regression maximize AUC? [duplicate]

$AUC$ is the area under the receiver operating characteristic curve. It's said that a loss function set to $1-AUC$ can be minimized by maximizing $AUC$. How does this tie into logit (logistic) ...
66 views

### Which performance metrics for highly imbalanced multiclass dataset?

I have a dataset with 5 classes. About 98% of the dataset belong to class 5. Classes 1-4 share equally about 2% of the dataset. However, it is highly important, that classes 1-4 are correctly ...
23 views

### Generating confidence interval for difference between 2 AUCs

I am trying to get a confidence interval for the difference between 2 AUCs for paired ROC curves. I've been reading the paper pROC: an open-source package for R and S+ to analyze and compare ROC ...
26 views

### Relationship between AUC of ROC curve and AUC of PR (precision-recall) curve

I know that both the ROC curve and the PR curve can be used to evaluate the performance of a binary classification prediction model, and PR curve is preferred in the case of imbalanced class ...
38 views

### Improving multiclass classification accuracy for LSTM network

I am working on time series multiclass classification task. My data consists of 5 features (3 categorical and 2 numerical features) and 3 classes target value. Here are the histograms of the class ...
16 views

### Evaluation metric for classification + regression, as in weighted True by another objective? AUC + MSE?

I'm using purchase data to build a cross-selling T/F response model for the banking industry, to score customers based on their likelihood to acquire a loan. On the other hand, I'm also building a ...
13 views

### What scoring metric is optimal when performing hyperparameter optimization with a multiclass target variable?

I'm trying to find the optimal hyperparameter for different algorithms where the target variable has 3 classes. I was wondering if maximizing the average AUC over the 3 classes, which I'm currently ...
31 views

### How to interpret an AUC value of 0.55

I have a classifier with an AUC value of 0.55. I understand that an AUC value of 0.5 is essentially a random classifier and not predictive at all. So I am wondering if an AUC of 0.55 is ...
25 views

### Why is an ROC curve TPR (Y) against FPR (X)?

I am trying to thoroughly understand the ROC Curve and I was wondering why is an ROC Curve always (seemingly) TPR against FPR? I have had discussions with others about this matter and I cannot think ...
36 views

### How to compare two or more mean AUC

I have different machine learning models and for each of them after a 10-fold cross validation I obtained a mean AUC (+-std). Now, How can I check if there is a statistically significant difference (...
8 views

### How to understand the mathematics of ROC/AUC in Wikipedia [duplicate]

Could someone help explain what happened in the red box? I'm struggling to figure out where the indicator function comes from in this calculation. Thank you~ wiki link
161 views

### R: Caret package: AUC (“ROC”) standard error from cross validation

This seems like a basic questions, so I'm likely missing the big picture here... I would like to know the AUC of each fold of a cross-validation performed in Caret's train function, in order to ...
33 views

### Adding predictors in ROC curves and how does it affect AUC?

I have a general question about ROC curves and how adding predictors affect AUC values. Let's say I have a model that contains only predictor A and produces an AUC of 0.6. I then add into the model a ...
42 views

### Calculating AUC on test set of random forest model in R

I have built a random forest model within a k-fold cv, and have had no problem calculating the AUC on the training set, but after trying a few methods of calculating AUC I keep getting errors to the ...
29 views

### calculate Specificity and sensitivity from AUC

can someone please tell me how do you calculate Specificity and sensitivity from AUC. thank you.
72 views

### ROC Convex Hull for Model Selection

The area under the convex hull of a roc curve is by construction always "better" than its area under curve. Some curves might see more of an increase in reported auc than others. Is it a viable ...
19 views

### Feature Selection by individual AUC

I am creating a model for classification and I have several ways to get subset of features but I was wondering if the following is reasonable: Use the train set to calculate LOOCV or LPOCV AUC values ...
47 views

### ROC Curve AUC for Hypothesis Testing Sensitivity (Power) vs Specificity ($1-\alpha$)

Let's set up a hypothesis test of $H_0: \theta=\theta_0$ versus $H_1: \theta\ne\theta_0$, and let's say that I have two techniques to assess this (say equal-variance t-test versus unequal-variance ...
109 views

### The usage of TOC curves?

In machine learning literature ROC curves are common performance measures. Quite recently published and not as popular (50+ citatations) are TOC curves as proposed by Pontius Jr, Robert Gilmore and Si,...
43 views

### Feature Selection - Overfit?

I have a dataset of 100 patients and 1500 features. I split 80 train 20 test first and then use the train set to get the best hyperparameters / best feature subset doing the following: I randomly ...