Questions tagged [roc]
Receiver Operating Characteristic, also known as the ROC curve, is a graphical plot that illustrates the diagnostic ability of a binary classifier system
834
questions
0
votes
1
answer
11
views
How is AUC helpful when we only need one threshold of a classifier
AUC is a summation of performance at different thresholds, but do we only care about a good performance at one threshold?
Imagine a classifier with a low ROC but shots up at point of a low FP and high ...
- 1
0
votes
0
answers
8
views
Area Under Precision-Recall and Area Under ROC curve for different amount of observations
I am doing a research and thus comparing some algorithms for binary classification. Worth to mention that, the data set is highly imbalanced i.e., the minority class is only 0.2%.
Notation:
Area Under ...
- 1
0
votes
0
answers
13
views
Cases where specificity can be very high and precision very low simultaneously?
I was trying to understand the difference between a ROC curve and a PR curve by reading this page: ROC vs precision-and-recall curves.
A quote from top voted the answer:
Interestingly, by Bayes' ...
- 869
0
votes
0
answers
12
views
Understand AUCs of different models
I'm testing two models against each other:
provides an AUC of 93,94 % with TP = 99,9 %, TN = 0 %, FP = 100 % and FN = 0.1 %.
provides an AUC of 92,78 % with TP = 98,8 %, a TN = 30,6 %, FP = 69,4 % ...
- 2,187
0
votes
0
answers
6
views
How to visualize ROC with output gotten in weka [closed]
Is there a way to visualize the ROC output given in WEKA using another software or method ? I have tried copying the ROC output values and plotting on excel but that gives a normal 2d graph.
I would ...
- 1
0
votes
0
answers
14
views
what is the acceptable value for acceptable cut point while minimizes distance between ROC plot and point (0,1)
If we would like to find the optimal cut point using minimizes distance between ROC plot and point (0,1), is it a acceptable value in the field for the roc01 value? I have searched for the reference ...
- 1
0
votes
0
answers
24
views
GLM; the function 'prediction' contains NA [closed]
I want to validate my glm and make a ROC curve.
I get some errors which I can't resolve.
My validation of the model:
...
0
votes
0
answers
29
views
What is my best option to evaluate a predictive (or proxy) variable?
I have a list of thunder flashes, and I'm trying to find if a meteorological variable (CAPE) is a good predictable variable (or proxy) for theses flashes. In my thoughts, I want to evaluate this proxy ...
- 1
0
votes
0
answers
23
views
Make ROC graph with my problematic [duplicate]
For a personal project, i'm trying to figure out how to trace some ROC/AUC graph with my current problematic.
I have a list of thunder flashes, and i'm trying to find if a meteorological variable (...
- 1
0
votes
0
answers
16
views
Determining cut-off (Youden) without causing data leakage
I´m new to coding and machine learning so this might sound like a stupid question.
I´m using Logistic regression, RF and SVM to model corporate defaults in R. However i´m a bit concerned of what could ...
- 1
0
votes
1
answer
16
views
How to interpret this confusion matrix and roc curve?
I got these two images for evaluating a RF:
I wonder why the ROC curve seems to be so good while the confusion matrix shows that the True Positive isn't so good with only ~16 %?
By looking only at ...
- 2,187
2
votes
1
answer
49
views
AUC - Logistic Regression versus LDA, and Naive Bayes
everyone!
I am a newbie on machine learning, and I am now interested on classification modeling.
I used logistic regression, linear discriminant analysis (LDA), and naive Bayes on my notebook DataCamp ...
0
votes
0
answers
9
views
Is there any Statistic method which could reflect the diagnose value when the prevalence of special type of characteristic is lower
Our present study is conducting a novel diagnosis test for predicting a disease by the X-ray. The gold-standard is a minimally invasive procedure can test a tissue sample for the disease.
In our ...
1
vote
1
answer
31
views
Logistic Regression: What is the value for precision when recall (true positive rate) is 0?
A quick overview of definitions before I get into the question:
True Positive (TP): An actual positive that the model classified as positive
False Positive (FP): An actual negative that the model ...
- 211
1
vote
1
answer
27
views
Is there a good reason for using AUROC on imbalanced dataset?
So I just learned about AUROC.
When I read this thread, it seems like AUROC is not a great metric for imbalanced dataset. One answer even says it shouldn't be used to compare models.
However, I am ...
- 135
3
votes
2
answers
59
views
Difference between ROC-AUC and Multiclass AUC (MAUC)
I am trying to understand the interpretation of these metrics in a multiclass scenario: ROC-AUC and MAUC. Scikit-learn provides ...
- 231
0
votes
0
answers
26
views
My data can be approximated with normal distribution (multimodal). How can I find the reasons and explain this behaviour?
I use DeLonge method to compare two ROC AUCS. The result of it is Z-score.
Both ROC AUCs obtained from LDA (linear discriminant analysis) from sklearn package. The ...
- 1
0
votes
0
answers
67
views
Precision-Recall curve and AUC in multi-class problems using R
I'm trying to evaluate a RandomForest model for multi-class classification using the Area Under Precision-Recall Curve. I need to plot the PR curve of each class and the micro and macro average and ...
- 113
0
votes
2
answers
73
views
Why does my ROC curve have a sharp edge?
I was working on a random forest model in R and I got a ROC curve that looks like this. This is very odd since there is no curvature. The data does have mostly qualitative features with only 2-3 ...
- 1
0
votes
0
answers
18
views
Name of the value when sensitivity=specificity
Is there a name for the value along the ROC curve where sensitivity=specificity? This seems like a reasonable way to have a single scalar value to compare classifiers.
- 91
0
votes
0
answers
16
views
Post test probability with a ROC curve
I have data that is normally distributed related to risk of a particular disease. At the median of the distribution, you would expect to observe the population prevalence level of disease P0=0.01. For ...
- 1
0
votes
0
answers
21
views
Compare three different algorithms for anomaly detection
I have 3 different anomaly detection algorithms, that I tested on a mock dataset of 5 elements.
The output of the first and second algorithms, that implement an LSTM, is true/false according to if ...
- 115
0
votes
1
answer
35
views
GEE - ROC Curves the Same Using Different Cluster Variables
I am creating multiple GEEs with the same covariates, but I am testing different clustering variables. The outcome is a binary yes/no variable and both VAR1 and VAR2 are binary, as well. Patients can ...
- 57
0
votes
0
answers
8
views
Applying ROC to evaluate time-series signal event picking model performance
I am trying to evaluate the performance of an event picking model that attempts to find the onset of a signal in a noisy time series.
Data contains the true signal time (ground truth) and the ...
3
votes
1
answer
78
views
Where does the Gini coefficient come from?
I understand what a ROC curve is. However, I do not understand the Gini coefficient in the context of binary classification.
All the resources I have checked state that $Gini = 1 - (2 \times AUC_{ROC})...
- 341
1
vote
0
answers
33
views
Thresholds and Cutoff Values Confusion
I am currently having confusion on a part in the paper:
Unal, Ilker. “Defining an Optimal Cut-Point Value in ROC Analysis: An Alternative Approach.” Computational and mathematical methods in medicine ...
1
vote
1
answer
71
views
What does high auc score but poor f1 indicate for imbalanced dataset?
I am working on a binary classification with an imbalanced dataset of 977 records (77:23 class ratio). My label 1 (POS not met) is the minority class.
Currently without any over/under sampling ...
- 1,748
0
votes
0
answers
30
views
How to compute ROC AUC for a method that uses two models?
To compare my method with others I'm trying to compute its AUC but I got a bit confused on how to do this for my case. My method uses a model that classifies an image as class A or B, after that if ...
2
votes
1
answer
48
views
Create AUC-ROC from single sensitivity and specificity value? [duplicate]
Is it possible and appropriate to estimate the area under the receiver operating characteristic curve from a single point estimate of an individual's sensitivity and specificity performance?
- 33
3
votes
1
answer
54
views
Extension of the relationship between ROCAUC/c-index and Wilcoxon-Mann-Whitney U
There is a relationship between ROCAUC and the Wilcoxon Mann-Whitney U test of the probability predictions of the two groups.
If our $y$ variable has more than two classes, can we extend this to a ...
- 30.9k
1
vote
2
answers
72
views
Proper way to finding area under ROC curve (AUC)
I have data of 200 persons with 2 variables:
1. C_level: Blood level of a chemical C (a numeric value)
2. Disease: Yes or No
I want to know if C_level can be used ...
- 8,993
0
votes
1
answer
69
views
How am I misunderstanding sklearn.metrics.roc_auc_score? [closed]
The documentation for scikit-learn sklearn.metrics.roc_auc_score() contains two statements regarding the 'average' parameter that, together, are confusing me:
Note: multiclass ROC AUC currently only ...
- 145
4
votes
4
answers
547
views
Validity of AUC for binary categorical variables
Scikit-learn function roc_auc_score can be used to get area under curve (AUC) of ROC curve. This score is generally used for numeric predictors' value in predicting ...
- 8,993
0
votes
0
answers
14
views
Roc curve straight line interpretation [duplicate]
How can I fix the straight line? is that mean that 70% of my cases that tag as positive and actually negative have the same value?
2
votes
1
answer
39
views
Is it valid to do ROC analysis without using test data?
I am trying to do an ROC analysis to check whether a particular biomarker has any diagnostic power for screening a condition. Since I have only small dataset of 100. I am not splitting the data into ...
- 811
3
votes
1
answer
500
views
Straight line in roc curve
Hi, what does the green straight line at the end of the plot indicate? Is there some problem with the model?
nay
5
votes
2
answers
52
views
When using ROC curves for WWII Radars, what was the TN?
One of the origins of ROC curves seems to be to compare radar systems in WWII (source). How did they actually compute the False Positive Rate when they didn't have an estimate for True Negatives?
If I ...
- 53
0
votes
0
answers
29
views
ROC AUC for average and geometric mean of two models
Suppose I have two binary classification models built on the same dataset.
One has ROC AUC of 0.86, another one has ROC AUC of 0.92.
What will be the ROC AUC for the average of two models and their ...
- 1
0
votes
0
answers
10
views
Prediction, concordance, ROC curves, and the c-statistic: when is it appropriate to use?
I have a study with 2 variables where one variable should hypothetically predict the other.
Var1 is a continuous biological measurement being categorized into a binomial: does the biological ...
- 517
0
votes
0
answers
22
views
Outputs of model evaluation function
I am working on a 3-class ML problem and I am in the phase where I have to write an "evaluation" function for my training dataset (i.e. function that performs a cross-validation method to ...
- 113
0
votes
0
answers
16
views
Does the imbalance in my data affect model quality in this case?
I have two datasets, used for classification task: training and validation one.
Both of them are equally imbalanced, with about 24% of target value equal 1 and 75% of target value equal 0.
I am using ...
- 113
0
votes
0
answers
8
views
Determining if a model is efficient using a ROC curve or how does an OvR Classifier Know a False Positive?
I'm trying to predict if a change to a file will result in a bug in software development. I was recommended to create an ROC curve to evaluate how effective my model is. However, my last contact with ...
- 227
0
votes
0
answers
9
views
Reference request for the analogue of ROC curve for positive/negative predictive values
The Receiver Operating Characteristic curve relates TPR with FPR, which are defined as TP/P and FP/N respectively. Has anyone studied a similar curve that relates precision (aka positive predictive ...
- 532
0
votes
0
answers
14
views
Stats set-up question: R returning impossible values when trying to set up the regression for ROC with multiple curves [duplicate]
I have been trying to plot some pilot data in ROC space but have run into some issue that I suspect is because I am fairly new at this. Specifically, I set my regression up like so:
...
0
votes
1
answer
37
views
R returning impossible values when trying to set up the regression for ROC with multiple curves
Image of roc data
Hello! I have been trying to plot some pilot data in ROC space but have run into some issue that I suspect is because I am fairly new at this. Specifically, I set my regression up ...
0
votes
0
answers
48
views
S-Shaped/Vertical ROC curve crossing the diagonal
I am running binary logistic regression and when the ROC curve is created this is the output (see below):
I believe it makes sense somewhat, as the model does have 100% specificity and 0% sensitivity ...
0
votes
0
answers
15
views
ROC table with integers vs non-integers
Currently I’m using optimal.cutpoints package in R to evaluate ELISA data. I created a data frame from this optimal cut points function that pulled the cut points and their respective sensitivity, ...
- 23
1
vote
2
answers
142
views
Training AUC below 0.5
I've trained a logistic regression using a small number of predictors - pseudo R-squared is only 0.1 but I have significant terms and a nice low p value for the model. However, even on its own ...
- 193
3
votes
1
answer
37
views
The different usage of ROC between diagnotic tests and machine learning
I am currently very confused about the ROC usage in diagnostic tests and machine learning.
In the scenario of medical diagnostic studies, many tutorial does not mention the data split procedure as ...
- 43
0
votes
0
answers
36
views
Area under the curve comparison: DeLong test vs CI
How is it possible that in some papers they report AUC values for example for 2 different biomarkers who are contained in each others' confidence interval YET are reported to be significantly ...
- 1