# Questions tagged [roc]

Receiver Operating Characteristic, also known as the ROC curve, is a graphical plot that illustrates the diagnostic ability of a binary classifier system

834 questions
Filter by
Sorted by
Tagged with
0 votes
1 answer
11 views

### How is AUC helpful when we only need one threshold of a classifier

AUC is a summation of performance at different thresholds, but do we only care about a good performance at one threshold? Imagine a classifier with a low ROC but shots up at point of a low FP and high ...
0 votes
0 answers
8 views

### Area Under Precision-Recall and Area Under ROC curve for different amount of observations

I am doing a research and thus comparing some algorithms for binary classification. Worth to mention that, the data set is highly imbalanced i.e., the minority class is only 0.2%. Notation: Area Under ...
0 votes
0 answers
13 views

### Cases where specificity can be very high and precision very low simultaneously?

I was trying to understand the difference between a ROC curve and a PR curve by reading this page: ROC vs precision-and-recall curves. A quote from top voted the answer: Interestingly, by Bayes' ...
• 869
0 votes
0 answers
12 views

### Understand AUCs of different models

I'm testing two models against each other: provides an AUC of 93,94 % with TP = 99,9 %, TN = 0 %, FP = 100 % and FN = 0.1 %. provides an AUC of 92,78 % with TP = 98,8 %, a TN = 30,6 %, FP = 69,4 % ...
• 2,187
0 votes
0 answers
6 views

### How to visualize ROC with output gotten in weka [closed]

Is there a way to visualize the ROC output given in WEKA using another software or method ? I have tried copying the ROC output values and plotting on excel but that gives a normal 2d graph. I would ...
0 votes
0 answers
14 views

### what is the acceptable value for acceptable cut point while minimizes distance between ROC plot and point (0,1)

If we would like to find the optimal cut point using minimizes distance between ROC plot and point (0,1), is it a acceptable value in the field for the roc01 value? I have searched for the reference ...
0 votes
0 answers
24 views

### GLM; the function 'prediction' contains NA [closed]

I want to validate my glm and make a ROC curve. I get some errors which I can't resolve. My validation of the model: ...
0 votes
0 answers
29 views

### What is my best option to evaluate a predictive (or proxy) variable?

I have a list of thunder flashes, and I'm trying to find if a meteorological variable (CAPE) is a good predictable variable (or proxy) for theses flashes. In my thoughts, I want to evaluate this proxy ...
0 votes
0 answers
23 views

### Make ROC graph with my problematic [duplicate]

For a personal project, i'm trying to figure out how to trace some ROC/AUC graph with my current problematic. I have a list of thunder flashes, and i'm trying to find if a meteorological variable (...
0 votes
0 answers
16 views

### Determining cut-off (Youden) without causing data leakage

I´m new to coding and machine learning so this might sound like a stupid question. I´m using Logistic regression, RF and SVM to model corporate defaults in R. However i´m a bit concerned of what could ...
• 1
0 votes
1 answer
16 views

### How to interpret this confusion matrix and roc curve?

I got these two images for evaluating a RF: I wonder why the ROC curve seems to be so good while the confusion matrix shows that the True Positive isn't so good with only ~16 %? By looking only at ...
• 2,187
2 votes
1 answer
49 views

### AUC - Logistic Regression versus LDA, and Naive Bayes

everyone! I am a newbie on machine learning, and I am now interested on classification modeling. I used logistic regression, linear discriminant analysis (LDA), and naive Bayes on my notebook DataCamp ...
0 votes
0 answers
9 views

### Is there any Statistic method which could reflect the diagnose value when the prevalence of special type of characteristic is lower

Our present study is conducting a novel diagnosis test for predicting a disease by the X-ray. The gold-standard is a minimally invasive procedure can test a tissue sample for the disease. In our ...
1 vote
1 answer
31 views

### Logistic Regression: What is the value for precision when recall (true positive rate) is 0?

A quick overview of definitions before I get into the question: True Positive (TP): An actual positive that the model classified as positive False Positive (FP): An actual negative that the model ...
• 211
1 vote
1 answer
27 views

### Is there a good reason for using AUROC on imbalanced dataset?

So I just learned about AUROC. When I read this thread, it seems like AUROC is not a great metric for imbalanced dataset. One answer even says it shouldn't be used to compare models. However, I am ...
• 135
3 votes
2 answers
59 views

### Difference between ROC-AUC and Multiclass AUC (MAUC)

I am trying to understand the interpretation of these metrics in a multiclass scenario: ROC-AUC and MAUC. Scikit-learn provides ...
• 231
0 votes
0 answers
26 views

### My data can be approximated with normal distribution (multimodal). How can I find the reasons and explain this behaviour?

I use DeLonge method to compare two ROC AUCS. The result of it is Z-score. Both ROC AUCs obtained from LDA (linear discriminant analysis) from sklearn package. The ...
0 votes
0 answers
67 views

### Precision-Recall curve and AUC in multi-class problems using R

I'm trying to evaluate a RandomForest model for multi-class classification using the Area Under Precision-Recall Curve. I need to plot the PR curve of each class and the micro and macro average and ...
• 113
0 votes
2 answers
73 views

### Why does my ROC curve have a sharp edge?

I was working on a random forest model in R and I got a ROC curve that looks like this. This is very odd since there is no curvature. The data does have mostly qualitative features with only 2-3 ...
0 votes
0 answers
18 views

### Name of the value when sensitivity=specificity

Is there a name for the value along the ROC curve where sensitivity=specificity? This seems like a reasonable way to have a single scalar value to compare classifiers.
0 votes
0 answers
16 views

### Post test probability with a ROC curve

I have data that is normally distributed related to risk of a particular disease. At the median of the distribution, you would expect to observe the population prevalence level of disease P0=0.01. For ...
0 votes
0 answers
21 views

### Compare three different algorithms for anomaly detection

I have 3 different anomaly detection algorithms, that I tested on a mock dataset of 5 elements. The output of the first and second algorithms, that implement an LSTM, is true/false according to if ...
• 115
0 votes
1 answer
35 views

### GEE - ROC Curves the Same Using Different Cluster Variables

I am creating multiple GEEs with the same covariates, but I am testing different clustering variables. The outcome is a binary yes/no variable and both VAR1 and VAR2 are binary, as well. Patients can ...
0 votes
0 answers
8 views

### Applying ROC to evaluate time-series signal event picking model performance

I am trying to evaluate the performance of an event picking model that attempts to find the onset of a signal in a noisy time series. Data contains the true signal time (ground truth) and the ...
3 votes
1 answer
78 views

• 30.9k
1 vote
2 answers
72 views

### Proper way to finding area under ROC curve (AUC)

I have data of 200 persons with 2 variables: 1. C_level: Blood level of a chemical C (a numeric value) 2. Disease: Yes or No I want to know if C_level can be used ...
• 8,993
0 votes
1 answer
69 views

### How am I misunderstanding sklearn.metrics.roc_auc_score? [closed]

The documentation for scikit-learn sklearn.metrics.roc_auc_score() contains two statements regarding the 'average' parameter that, together, are confusing me: Note: multiclass ROC AUC currently only ...
• 145
4 votes
4 answers
547 views

### Validity of AUC for binary categorical variables

Scikit-learn function roc_auc_score can be used to get area under curve (AUC) of ROC curve. This score is generally used for numeric predictors' value in predicting ...
• 8,993
0 votes
0 answers
14 views

### Roc curve straight line interpretation [duplicate]

How can I fix the straight line? is that mean that 70% of my cases that tag as positive and actually negative have the same value?
2 votes
1 answer
39 views

### Is it valid to do ROC analysis without using test data?

I am trying to do an ROC analysis to check whether a particular biomarker has any diagnostic power for screening a condition. Since I have only small dataset of 100. I am not splitting the data into ...
• 811
3 votes
1 answer
500 views

### Straight line in roc curve

Hi, what does the green straight line at the end of the plot indicate? Is there some problem with the model?
5 votes
2 answers
52 views

### When using ROC curves for WWII Radars, what was the TN?

One of the origins of ROC curves seems to be to compare radar systems in WWII (source). How did they actually compute the False Positive Rate when they didn't have an estimate for True Negatives? If I ...
• 53
0 votes
0 answers
29 views

### ROC AUC for average and geometric mean of two models

Suppose I have two binary classification models built on the same dataset. One has ROC AUC of 0.86, another one has ROC AUC of 0.92. What will be the ROC AUC for the average of two models and their ...
0 votes
0 answers
10 views

### Prediction, concordance, ROC curves, and the c-statistic: when is it appropriate to use?

I have a study with 2 variables where one variable should hypothetically predict the other. Var1 is a continuous biological measurement being categorized into a binomial: does the biological ...
• 517
0 votes
0 answers
22 views

### Outputs of model evaluation function

I am working on a 3-class ML problem and I am in the phase where I have to write an "evaluation" function for my training dataset (i.e. function that performs a cross-validation method to ...
• 113
0 votes
0 answers
16 views

### Does the imbalance in my data affect model quality in this case?

I have two datasets, used for classification task: training and validation one. Both of them are equally imbalanced, with about 24% of target value equal 1 and 75% of target value equal 0. I am using ...
• 113
0 votes
0 answers
8 views

### Determining if a model is efficient using a ROC curve or how does an OvR Classifier Know a False Positive?

I'm trying to predict if a change to a file will result in a bug in software development. I was recommended to create an ROC curve to evaluate how effective my model is. However, my last contact with ...
0 votes
0 answers
9 views

### Reference request for the analogue of ROC curve for positive/negative predictive values

The Receiver Operating Characteristic curve relates TPR with FPR, which are defined as TP/P and FP/N respectively. Has anyone studied a similar curve that relates precision (aka positive predictive ...
• 532
0 votes
0 answers
14 views

### Stats set-up question: R returning impossible values when trying to set up the regression for ROC with multiple curves [duplicate]

I have been trying to plot some pilot data in ROC space but have run into some issue that I suspect is because I am fairly new at this. Specifically, I set my regression up like so: ...
0 votes
1 answer
37 views

### R returning impossible values when trying to set up the regression for ROC with multiple curves

Image of roc data Hello! I have been trying to plot some pilot data in ROC space but have run into some issue that I suspect is because I am fairly new at this. Specifically, I set my regression up ...
0 votes
0 answers
48 views

### S-Shaped/Vertical ROC curve crossing the diagonal

I am running binary logistic regression and when the ROC curve is created this is the output (see below): I believe it makes sense somewhat, as the model does have 100% specificity and 0% sensitivity ...
0 votes
0 answers
15 views

### ROC table with integers vs non-integers

Currently I’m using optimal.cutpoints package in R to evaluate ELISA data. I created a data frame from this optimal cut points function that pulled the cut points and their respective sensitivity, ...
1 vote
2 answers
142 views

### Training AUC below 0.5

I've trained a logistic regression using a small number of predictors - pseudo R-squared is only 0.1 but I have significant terms and a nice low p value for the model. However, even on its own ...
• 193
3 votes
1 answer
37 views

### The different usage of ROC between diagnotic tests and machine learning

I am currently very confused about the ROC usage in diagnostic tests and machine learning. In the scenario of medical diagnostic studies, many tutorial does not mention the data split procedure as ...
• 43
0 votes
0 answers
36 views

### Area under the curve comparison: DeLong test vs CI

How is it possible that in some papers they report AUC values for example for 2 different biomarkers who are contained in each others' confidence interval YET are reported to be significantly ...
• 1