Questions tagged [roc]

Receiver Operating Characteristic, also known as the ROC curve, is a graphical plot that illustrates the diagnostic ability of a binary classifier system

Filter by
Sorted by
Tagged with
0 votes
1 answer
11 views

How is AUC helpful when we only need one threshold of a classifier

AUC is a summation of performance at different thresholds, but do we only care about a good performance at one threshold? Imagine a classifier with a low ROC but shots up at point of a low FP and high ...
user avatar
0 votes
0 answers
8 views

Area Under Precision-Recall and Area Under ROC curve for different amount of observations

I am doing a research and thus comparing some algorithms for binary classification. Worth to mention that, the data set is highly imbalanced i.e., the minority class is only 0.2%. Notation: Area Under ...
user avatar
  • 1
0 votes
0 answers
13 views

Cases where specificity can be very high and precision very low simultaneously?

I was trying to understand the difference between a ROC curve and a PR curve by reading this page: ROC vs precision-and-recall curves. A quote from top voted the answer: Interestingly, by Bayes' ...
user avatar
  • 869
0 votes
0 answers
12 views

Understand AUCs of different models

I'm testing two models against each other: provides an AUC of 93,94 % with TP = 99,9 %, TN = 0 %, FP = 100 % and FN = 0.1 %. provides an AUC of 92,78 % with TP = 98,8 %, a TN = 30,6 %, FP = 69,4 % ...
user avatar
  • 2,187
0 votes
0 answers
6 views

How to visualize ROC with output gotten in weka [closed]

Is there a way to visualize the ROC output given in WEKA using another software or method ? I have tried copying the ROC output values and plotting on excel but that gives a normal 2d graph. I would ...
user avatar
0 votes
0 answers
14 views

what is the acceptable value for acceptable cut point while minimizes distance between ROC plot and point (0,1)

If we would like to find the optimal cut point using minimizes distance between ROC plot and point (0,1), is it a acceptable value in the field for the roc01 value? I have searched for the reference ...
user avatar
  • 1
0 votes
0 answers
24 views

GLM; the function 'prediction' contains NA [closed]

I want to validate my glm and make a ROC curve. I get some errors which I can't resolve. My validation of the model: ...
user avatar
0 votes
0 answers
29 views

What is my best option to evaluate a predictive (or proxy) variable?

I have a list of thunder flashes, and I'm trying to find if a meteorological variable (CAPE) is a good predictable variable (or proxy) for theses flashes. In my thoughts, I want to evaluate this proxy ...
user avatar
0 votes
0 answers
23 views

Make ROC graph with my problematic [duplicate]

For a personal project, i'm trying to figure out how to trace some ROC/AUC graph with my current problematic. I have a list of thunder flashes, and i'm trying to find if a meteorological variable (...
user avatar
0 votes
0 answers
16 views

Determining cut-off (Youden) without causing data leakage

I´m new to coding and machine learning so this might sound like a stupid question. I´m using Logistic regression, RF and SVM to model corporate defaults in R. However i´m a bit concerned of what could ...
user avatar
  • 1
0 votes
1 answer
16 views

How to interpret this confusion matrix and roc curve?

I got these two images for evaluating a RF: I wonder why the ROC curve seems to be so good while the confusion matrix shows that the True Positive isn't so good with only ~16 %? By looking only at ...
user avatar
  • 2,187
2 votes
1 answer
49 views

AUC - Logistic Regression versus LDA, and Naive Bayes

everyone! I am a newbie on machine learning, and I am now interested on classification modeling. I used logistic regression, linear discriminant analysis (LDA), and naive Bayes on my notebook DataCamp ...
user avatar
0 votes
0 answers
9 views

Is there any Statistic method which could reflect the diagnose value when the prevalence of special type of characteristic is lower

Our present study is conducting a novel diagnosis test for predicting a disease by the X-ray. The gold-standard is a minimally invasive procedure can test a tissue sample for the disease. In our ...
user avatar
1 vote
1 answer
31 views

Logistic Regression: What is the value for precision when recall (true positive rate) is 0?

A quick overview of definitions before I get into the question: True Positive (TP): An actual positive that the model classified as positive False Positive (FP): An actual negative that the model ...
user avatar
  • 211
1 vote
1 answer
27 views

Is there a good reason for using AUROC on imbalanced dataset?

So I just learned about AUROC. When I read this thread, it seems like AUROC is not a great metric for imbalanced dataset. One answer even says it shouldn't be used to compare models. However, I am ...
user avatar
  • 135
3 votes
2 answers
59 views

Difference between ROC-AUC and Multiclass AUC (MAUC)

I am trying to understand the interpretation of these metrics in a multiclass scenario: ROC-AUC and MAUC. Scikit-learn provides ...
user avatar
  • 231
0 votes
0 answers
26 views

My data can be approximated with normal distribution (multimodal). How can I find the reasons and explain this behaviour?

I use DeLonge method to compare two ROC AUCS. The result of it is Z-score. Both ROC AUCs obtained from LDA (linear discriminant analysis) from sklearn package. The ...
user avatar
0 votes
0 answers
67 views

Precision-Recall curve and AUC in multi-class problems using R

I'm trying to evaluate a RandomForest model for multi-class classification using the Area Under Precision-Recall Curve. I need to plot the PR curve of each class and the micro and macro average and ...
user avatar
  • 113
0 votes
2 answers
73 views

Why does my ROC curve have a sharp edge?

I was working on a random forest model in R and I got a ROC curve that looks like this. This is very odd since there is no curvature. The data does have mostly qualitative features with only 2-3 ...
user avatar
  • 1
0 votes
0 answers
18 views

Name of the value when sensitivity=specificity

Is there a name for the value along the ROC curve where sensitivity=specificity? This seems like a reasonable way to have a single scalar value to compare classifiers.
user avatar
0 votes
0 answers
16 views

Post test probability with a ROC curve

I have data that is normally distributed related to risk of a particular disease. At the median of the distribution, you would expect to observe the population prevalence level of disease P0=0.01. For ...
user avatar
0 votes
0 answers
21 views

Compare three different algorithms for anomaly detection

I have 3 different anomaly detection algorithms, that I tested on a mock dataset of 5 elements. The output of the first and second algorithms, that implement an LSTM, is true/false according to if ...
user avatar
  • 115
0 votes
1 answer
35 views

GEE - ROC Curves the Same Using Different Cluster Variables

I am creating multiple GEEs with the same covariates, but I am testing different clustering variables. The outcome is a binary yes/no variable and both VAR1 and VAR2 are binary, as well. Patients can ...
user avatar
0 votes
0 answers
8 views

Applying ROC to evaluate time-series signal event picking model performance

I am trying to evaluate the performance of an event picking model that attempts to find the onset of a signal in a noisy time series. Data contains the true signal time (ground truth) and the ...
user avatar
3 votes
1 answer
78 views

Where does the Gini coefficient come from?

I understand what a ROC curve is. However, I do not understand the Gini coefficient in the context of binary classification. All the resources I have checked state that $Gini = 1 - (2 \times AUC_{ROC})...
user avatar
1 vote
0 answers
33 views

Thresholds and Cutoff Values Confusion

I am currently having confusion on a part in the paper: Unal, Ilker. “Defining an Optimal Cut-Point Value in ROC Analysis: An Alternative Approach.” Computational and mathematical methods in medicine ...
user avatar
1 vote
1 answer
71 views

What does high auc score but poor f1 indicate for imbalanced dataset?

I am working on a binary classification with an imbalanced dataset of 977 records (77:23 class ratio). My label 1 (POS not met) is the minority class. Currently without any over/under sampling ...
user avatar
  • 1,748
0 votes
0 answers
30 views

How to compute ROC AUC for a method that uses two models?

To compare my method with others I'm trying to compute its AUC but I got a bit confused on how to do this for my case. My method uses a model that classifies an image as class A or B, after that if ...
user avatar
2 votes
1 answer
48 views

Create AUC-ROC from single sensitivity and specificity value? [duplicate]

Is it possible and appropriate to estimate the area under the receiver operating characteristic curve from a single point estimate of an individual's sensitivity and specificity performance?
user avatar
  • 33
3 votes
1 answer
54 views

Extension of the relationship between ROCAUC/c-index and Wilcoxon-Mann-Whitney U

There is a relationship between ROCAUC and the Wilcoxon Mann-Whitney U test of the probability predictions of the two groups. If our $y$ variable has more than two classes, can we extend this to a ...
user avatar
  • 30.9k
1 vote
2 answers
72 views

Proper way to finding area under ROC curve (AUC)

I have data of 200 persons with 2 variables: 1. C_level: Blood level of a chemical C (a numeric value) 2. Disease: Yes or No I want to know if C_level can be used ...
user avatar
  • 8,993
0 votes
1 answer
69 views

How am I misunderstanding sklearn.metrics.roc_auc_score? [closed]

The documentation for scikit-learn sklearn.metrics.roc_auc_score() contains two statements regarding the 'average' parameter that, together, are confusing me: Note: multiclass ROC AUC currently only ...
user avatar
  • 145
4 votes
4 answers
547 views

Validity of AUC for binary categorical variables

Scikit-learn function roc_auc_score can be used to get area under curve (AUC) of ROC curve. This score is generally used for numeric predictors' value in predicting ...
user avatar
  • 8,993
0 votes
0 answers
14 views

Roc curve straight line interpretation [duplicate]

How can I fix the straight line? is that mean that 70% of my cases that tag as positive and actually negative have the same value?
user avatar
2 votes
1 answer
39 views

Is it valid to do ROC analysis without using test data?

I am trying to do an ROC analysis to check whether a particular biomarker has any diagnostic power for screening a condition. Since I have only small dataset of 100. I am not splitting the data into ...
user avatar
  • 811
3 votes
1 answer
500 views

Straight line in roc curve

Hi, what does the green straight line at the end of the plot indicate? Is there some problem with the model?
user avatar
5 votes
2 answers
52 views

When using ROC curves for WWII Radars, what was the TN?

One of the origins of ROC curves seems to be to compare radar systems in WWII (source). How did they actually compute the False Positive Rate when they didn't have an estimate for True Negatives? If I ...
user avatar
  • 53
0 votes
0 answers
29 views

ROC AUC for average and geometric mean of two models

Suppose I have two binary classification models built on the same dataset. One has ROC AUC of 0.86, another one has ROC AUC of 0.92. What will be the ROC AUC for the average of two models and their ...
user avatar
0 votes
0 answers
10 views

Prediction, concordance, ROC curves, and the c-statistic: when is it appropriate to use?

I have a study with 2 variables where one variable should hypothetically predict the other. Var1 is a continuous biological measurement being categorized into a binomial: does the biological ...
user avatar
  • 517
0 votes
0 answers
22 views

Outputs of model evaluation function

I am working on a 3-class ML problem and I am in the phase where I have to write an "evaluation" function for my training dataset (i.e. function that performs a cross-validation method to ...
user avatar
  • 113
0 votes
0 answers
16 views

Does the imbalance in my data affect model quality in this case?

I have two datasets, used for classification task: training and validation one. Both of them are equally imbalanced, with about 24% of target value equal 1 and 75% of target value equal 0. I am using ...
user avatar
  • 113
0 votes
0 answers
8 views

Determining if a model is efficient using a ROC curve or how does an OvR Classifier Know a False Positive?

I'm trying to predict if a change to a file will result in a bug in software development. I was recommended to create an ROC curve to evaluate how effective my model is. However, my last contact with ...
user avatar
0 votes
0 answers
9 views

Reference request for the analogue of ROC curve for positive/negative predictive values

The Receiver Operating Characteristic curve relates TPR with FPR, which are defined as TP/P and FP/N respectively. Has anyone studied a similar curve that relates precision (aka positive predictive ...
user avatar
0 votes
0 answers
14 views

Stats set-up question: R returning impossible values when trying to set up the regression for ROC with multiple curves [duplicate]

I have been trying to plot some pilot data in ROC space but have run into some issue that I suspect is because I am fairly new at this. Specifically, I set my regression up like so: ...
user avatar
0 votes
1 answer
37 views

R returning impossible values when trying to set up the regression for ROC with multiple curves

Image of roc data Hello! I have been trying to plot some pilot data in ROC space but have run into some issue that I suspect is because I am fairly new at this. Specifically, I set my regression up ...
user avatar
0 votes
0 answers
48 views

S-Shaped/Vertical ROC curve crossing the diagonal

I am running binary logistic regression and when the ROC curve is created this is the output (see below): I believe it makes sense somewhat, as the model does have 100% specificity and 0% sensitivity ...
user avatar
0 votes
0 answers
15 views

ROC table with integers vs non-integers

Currently I’m using optimal.cutpoints package in R to evaluate ELISA data. I created a data frame from this optimal cut points function that pulled the cut points and their respective sensitivity, ...
user avatar
1 vote
2 answers
142 views

Training AUC below 0.5

I've trained a logistic regression using a small number of predictors - pseudo R-squared is only 0.1 but I have significant terms and a nice low p value for the model. However, even on its own ...
user avatar
3 votes
1 answer
37 views

The different usage of ROC between diagnotic tests and machine learning

I am currently very confused about the ROC usage in diagnostic tests and machine learning. In the scenario of medical diagnostic studies, many tutorial does not mention the data split procedure as ...
user avatar
  • 43
0 votes
0 answers
36 views

Area under the curve comparison: DeLong test vs CI

How is it possible that in some papers they report AUC values for example for 2 different biomarkers who are contained in each others' confidence interval YET are reported to be significantly ...
user avatar
  • 1

1
2 3 4 5
17