# Questions tagged [roc]

Receiver Operating Characteristic, also known as the ROC curve.

647 questions
Filter by
Sorted by
Tagged with
23 views

### AUC in scikit-learn [closed]

I've trained a binary classifier in scikit-learn and I get two different results for AUC when calculating on the test data: 1- I use sklearn.metrics.auc to calculate and I get 0.63 2- I plot the ROC ...
17 views

### How to properly report confidence from distance/threshold (face detection)

In the context of face matching I have the following histogram: blue bins count the comparison distances for "self matches" (comparing two images of the same person). Orange bins count the distances ...
868 views

### Dichotomizing continuous variables at their optimal cut-off for clinical interpretation

In medical context, when presenting results from a binary outcome with a continuous predictor, the OR (odds ratio) can be difficult to interpret. Example: A doctor does a study in which he wants to ...
11 views

### Can I ignore multicolinearity if i am using predicted probabilities from a logistic regression as inputs into a ROC curve?

I have 8 variables, 6 of which are highly correlated with one another. I want to see how well these variables all together than classify a known binary grouping. I therefore want to run a multiple ...
30 views

### What is worst than a random, in ROC then say lower-right space is worst than random

Nothing can be worst than random, if you are predicting wrong all the time then why don't you just invert your results all the time and then you will be in the upper-left area of the ROC curve i.e. ...
25 views

### ROC curves: can a cut-point (cut-off) be “useful”, or is it a term reserved to parameters only?

This is a semantic question: When analyzing Receiver Operating Characteric (ROC) curves, we talk about the "usefulness" of a parameter based on its Area Under the Curve (AUC). "Useful" here refers to ...
37 views

### ROC curve looking off compared to my results

I have a dependent variable "response" as binary 1 = response, 0 = no response (for surgery). I have an independent variable of a certain measurement in degrees (continuous/ordered variable). A ...
33 views

### PR AUC vs ROC AUC for imbalanced data [duplicate]

I am struggling with choosing metric that I will use to compare models performance and hiperparam search. My task is similar to fraud detection. I have found out that many people states PR is better ...
10 views

### The results of test set are better than train set. why? [duplicate]

I have made a comparison of logit, ddhazard, decision tree and random forest models on ROC curve. the results of test sample are better than train set. why?
40 views

### How to calculate TPR and FPR and plot ROC curves for object detection?

According to its Wikipedia page, receiver operating curves are created by plotting the TPR vs. the FPR at various discrimination thresholds where: ...
8 views

### What is threshold in ROC curve? [duplicate]

Whenever I read about ROC, people say that it is graphical representation of True Positive Rate value and False Positive Rate at various threshold. Whenever I read in detail, people explain that ...
37 views

### Tuning threshold from multiclass ROC for Gradient Boosting Classifier?

I have created a ROC curve based on the output of a multiclass Gradient Boosting Classifier (See Figure below implemented from Yellowbrick ROCAUC: https://www.scikit-yb.org/en/latest/api/classifier/...
32 views

### ROC curve under diagonal?

I trained an SVM to classify images based on some extracted features (using the ISIC dataset). The resulting ROC curve produced by sklearn looks like this: I have don't quite understand the line for ...
20 views

### How high is multiclass AUROC too high?

Whenever I get AUROC above 80% for a binary classification problem I do my best to check for leakage and overfitting - and usually my intuition is right, true AUROC is closer to the 70%-75% range. ...
48 views

### Why is the mean of sensitivity and specificity equal to the AUC?

For a given cut-point in a prediction model or score, the mean of sensitivity and specificity equals the AUC. I've read that and I have observed this empirically. How can I prove this?
5 views

### Minimum proportion of negatives in ROC test?

apologies for the basic question, but are there any generally accepted guidelines on the minimum proportion of negative to positive outcomes required for ROC analysis? For example, I have data where ...
32 views

### Statistical analysis in R language

I have a few questions about statistical tests in R. I have a dependent variable Admitted, whose values could be either Yes or No and have 100 observations. When we train our model, we get results ...
40 views

### How to plot multiple classifiers' ROC curves using scikitplot?

I have classified a data with multiple classes (not binary) by using several classifiers, and I would like to compare the performance of these classifiers by drawing their ROC curves using scikitplot. ...
6 views

### Left-Truncated Right-Censored Cox Proportional Hazard Regression

If I have a left-truncated (remove everyone who has disease at baseline) and right-censored (1 if the person develops disease, 0 else). Time to event = min(age censored, age at disease diagnosis) ...
14 views

### Unrealistically high AUC-ROC score comparing to control feature and other performance measures

I am making a binary classification using regularized logistic regression, with extreme unbalanced data. The target label is Tar and non-target label is ...
25 views

### ROC score for binary classification problem, where the predictions are either 0 or 1

For problems with binary classification, roc auc curve or roc auc score is often used to rate a model. But does the ROC ACC make sense in the context of a binary classification model that outputs only ...
10 views

### How do i evaluate my one hot encoded multi dimensional predictions?

fellows, I am using keras (LSTM) to predict one hot encoded set of categories. My categories look like this: ...
27 views

### non-significant coefficient in logistic regression, use for ROC curve

I wanted to find the ability of a numeric continuous variable to predict mortality (dead/alive), and what cut off value to take in the continuous variable. I wanted to construct an ROC curve and find ...
11 views

### ROC for time-varying covariates in ddhazard model?

i want to draw ROC for ddhazard model. I have tried risksetROC package. But i am unable to get predictors for my model. here is the code: OBTAIN A LINEAR PREDICTOR predict1<- predict(ddfit1,type =...
41 views

### How to calculate and plot cross-validated ROC?

I am doing K-fold cross validation and I want to plot an averaged ROC curve in MATLAB. However currently I can only plot K ROC curves in one plot but without knowing the algorithm of averaged ROC ...
44 views

### Power and sample size calculations for unpaired ROC curves

I would like to perform power and sample size calculations for comparison of unpaired receiver-operating characteristic (ROC) curves. I have tried using the power.roc.test function from pROC package ...
26 views

### R-Caret : Not-meaningful class probabilities and AUC value

I am very new to ML therefore my question might be primitive. I am working on a binary-class problem. The response (target) variable is occurrence : a factor ...
29 views

### Finding optimal cutoff point [duplicate]

.Hello,everyone. I am studying the influence of one biomarker on multiple disease characteristics, and I would like to calculate its cutoff point. I created univariate ROC curves to investigate the ...
25 views

### Possible reasons for AUC=1 (from a fitted glm model)?

I am running high-throughput microarray data (methylation array), and after running univariate, lasso and cross-validation lasso analyses, I was able to come down to a list of 15 probes (predictors). ...
59 views

### ROC Curve Horizontal Line

I'm trying to conceptually understand 'why' the horizontal gray line in ROC curve below would be considered random. I know the basics of the ROC in that the farther the curved black line is from the ...
36 views

### Calculate AUC using predicted values and labels from a 5 fold classification?

I have a classifier for a binary problem. That has outputs between 0 and 1 for predictions for the two class A or B (for example sunny, not sunny). The classifier has ran on 5 unique folds of the data ...
53 views

### How to decide threshold values for AUC?

With respect to the pROC package for R(https://rdrr.io/cran/pROC/man/ggroc.html). ...
20 views

### Prove that AUROC is an improper scoring rule [duplicate]

It has been stated in many places that AUROC is an improper scoring rule.But I haven't seen anyone proving it. Does someone have a working example that shows that maximizing AUROC actually moves away ...
47 views

### RandomForest ROC curve [closed]

How can I draw the roc curve of this model? ...
29 views

### When do ROC-Curves intersect?

In books I see ROC curves like this: So I always think the model with the highest AUC is better at any threshold. Is this true in general? When applying usual classification models, can there be ...
166 views

### glmStepAIC model is doing better that other models

I am training a model on an imbalanced dataset (about 5-20% of positive class) and trying out different algorithms in R using caret package. I have 57 predictors and around 2000-3000 observations in ...
281 views

### Calculate AUC using sensitivity and specificity values only

How to calculate AUC, if I have values of sensitivity and specificity for various threshold cutoffs? I have sensitivity and specificity values for 100 thresholds. ...
25 views

### 3D ROC surface from multinomial logistic regression with more than one predictor

How do you construct a 3D ROC surface in R from a multinomial logistic regression? I have been able to form a 3D ROC surface for the individual models within a multinomial logistic regression, but not ...
62 views

### Compare two Roc curves with same Auc

I have 2 churn prediction model. Both provided very similar AUC values for Roc but with different shape. How should I assess which to choose based on that fact?
165 views

### PR AUC < 50% with ROC AUC > 90% - model good or bad?

I understand for highly imbalanced dataset - we need to look for precision-recall vs ROC AUC to better judge the model. My question is what is the range for PR AUC below which the model is bad? My ...
18 views

### AUC-like measure for multiple simultaneous classification tasks?

I know that given an ordered set of binary labels, and equally-sized ordered set of scalar predictions, we can quantify how cleanly the predictions separate the labels into clean buckets of 0's and 1'...
54 views

### ROC-style curves for calculating sample size, power, alpha, and effect size

I found an awesome R package called pwr that does all sorts of calculations about sample sizes, power, effect sizes, and so on, and I've been playing. I have a ...
30 views

### ROC curve for a perfect classifier

I have trained a DNN classifier on a balanced dataset which outputs softmax values $p_1$ and $p_0$ corresponding to probabilities of positive (label 1) and negative (label 0) classes, respectively. I ...
126 views

### Generate synthetic data given AUC

I'm experimenting with ROC-AUC for binary classification problems. I want to generate synthetic data for a given AUC score. The ...
79 views

### Statistical evidence that the AUC was not overfitted to the model. With N=119, C-stat = 0.81 seems optimistic. Optimism-adjusted?

My data have 119 cases and we did ROC for x (continuous variable) to predict postoperative y (categorical variable) available here, we got a comment from a reviewer asking: Please provide ...
14 views

### Statistical evidence that the AUC was not overfitted to the model. With N=119, C-stat = 0.81 seems optimistic. Optimism-adjusted? [duplicate]

My data has 119 case and We did ROC for x (continuous variable ) to predict postoperative y (categorical variable) available here, we got a comment from a reviewer asking "Please provide statistical ...
24 views

### Area under the ROC curve accuracy [duplicate]

Why is the area under the ROC curve better than raw accuracy as an out-of- sample evaluation metric?
55 views

### SVM: achieving ROC curve by varying misclassification costs

Assume that we have SVM model for binary classification with objective function as follows:  min(\frac{1}{2}\omega\cdot\omega +C^{+}\sum_{i|y_{i}=+1}^{n}\xi_{i}\quad+C^{-}\sum_{i|y_{i}=-1}^{n}\xi_{i}...