Questions tagged [roc]
Receiver Operating Characteristic, also known as the ROC curve.
647
questions
0
votes
0answers
23 views
AUC in scikit-learn [closed]
I've trained a binary classifier in scikit-learn and I get two different results for AUC when calculating on the test data:
1- I use sklearn.metrics.auc to calculate and I get 0.63
2- I plot the ROC ...
0
votes
0answers
17 views
How to properly report confidence from distance/threshold (face detection)
In the context of face matching I have the following histogram:
blue bins count the comparison distances for "self matches" (comparing two images of the same person). Orange bins count the distances ...
13
votes
4answers
868 views
Dichotomizing continuous variables at their optimal cut-off for clinical interpretation
In medical context, when presenting results from a binary outcome with a continuous predictor, the OR (odds ratio) can be difficult to interpret.
Example:
A doctor does a study in which he wants to ...
0
votes
0answers
11 views
Can I ignore multicolinearity if i am using predicted probabilities from a logistic regression as inputs into a ROC curve?
I have 8 variables, 6 of which are highly correlated with one another. I want to see how well these variables all together than classify a known binary grouping. I therefore want to run a multiple ...
1
vote
3answers
30 views
What is worst than a random, in ROC then say lower-right space is worst than random
Nothing can be worst than random, if you are predicting wrong all the time then why don't you just invert your results all the time and then you will be in the upper-left area of the ROC curve i.e. ...
1
vote
1answer
25 views
ROC curves: can a cut-point (cut-off) be “useful”, or is it a term reserved to parameters only?
This is a semantic question:
When analyzing Receiver Operating Characteric (ROC) curves, we talk about the "usefulness" of a parameter based on its Area Under the Curve (AUC). "Useful" here refers to ...
1
vote
1answer
37 views
ROC curve looking off compared to my results
I have a dependent variable "response" as binary 1 = response, 0 = no response (for surgery). I have an independent variable of a certain measurement in degrees (continuous/ordered variable).
A ...
1
vote
0answers
33 views
PR AUC vs ROC AUC for imbalanced data [duplicate]
I am struggling with choosing metric that I will use to compare models performance and hiperparam search. My task is similar to fraud detection.
I have found out that many people states PR is better ...
0
votes
0answers
10 views
The results of test set are better than train set. why? [duplicate]
I have made a comparison of logit, ddhazard, decision tree and random forest models on ROC curve. the results of test sample are better than train set. why?
1
vote
0answers
40 views
How to calculate TPR and FPR and plot ROC curves for object detection?
According to its Wikipedia page, receiver operating curves are created by plotting the TPR vs. the FPR at various discrimination thresholds where:
...
0
votes
0answers
8 views
What is threshold in ROC curve? [duplicate]
Whenever I read about ROC, people say that it is graphical representation of True Positive Rate value and False Positive Rate at various threshold.
Whenever I read in detail, people explain that ...
0
votes
0answers
37 views
Tuning threshold from multiclass ROC for Gradient Boosting Classifier?
I have created a ROC curve based on the output of a multiclass Gradient Boosting Classifier (See Figure below implemented from Yellowbrick ROCAUC: https://www.scikit-yb.org/en/latest/api/classifier/...
0
votes
0answers
32 views
ROC curve under diagonal?
I trained an SVM to classify images based on some extracted features (using the ISIC dataset). The resulting ROC curve produced by sklearn looks like this:
I have don't quite understand the line for ...
0
votes
0answers
20 views
How high is multiclass AUROC too high?
Whenever I get AUROC above 80% for a binary classification problem I do my best to check for leakage and overfitting - and usually my intuition is right, true AUROC is closer to the 70%-75% range.
...
1
vote
2answers
48 views
Why is the mean of sensitivity and specificity equal to the AUC?
For a given cut-point in a prediction model or score, the mean of sensitivity and specificity equals the AUC. I've read that and I have observed this empirically.
How can I prove this?
0
votes
0answers
5 views
Minimum proportion of negatives in ROC test?
apologies for the basic question, but are there any generally accepted guidelines on the minimum proportion of negative to positive outcomes required for ROC analysis?
For example, I have data where ...
0
votes
0answers
32 views
Statistical analysis in R language
I have a few questions about statistical tests in R. I have a dependent variable Admitted, whose values could be either Yes or No and have 100 observations. When we train our model, we get results ...
0
votes
0answers
40 views
How to plot multiple classifiers' ROC curves using scikitplot?
I have classified a data with multiple classes (not binary) by using several classifiers, and I would like to compare the performance of these classifiers by drawing their ROC curves using scikitplot.
...
0
votes
0answers
6 views
Left-Truncated Right-Censored Cox Proportional Hazard Regression
If I have a left-truncated (remove everyone who has disease at baseline) and right-censored (1 if the person develops disease, 0 else).
Time to event = min(age censored, age at disease diagnosis)
...
0
votes
0answers
14 views
Unrealistically high AUC-ROC score comparing to control feature and other performance measures
I am making a binary classification using regularized logistic regression, with extreme unbalanced data. The target label is Tar and non-target label is ...
0
votes
1answer
25 views
ROC score for binary classification problem, where the predictions are either 0 or 1
For problems with binary classification, roc auc curve or roc auc score is often used to rate a model. But does the ROC ACC make sense in the context of a binary classification model that outputs only ...
0
votes
0answers
10 views
How do i evaluate my one hot encoded multi dimensional predictions?
fellows, I am using keras (LSTM) to predict one hot encoded set of categories.
My categories look like this:
...
0
votes
2answers
27 views
non-significant coefficient in logistic regression, use for ROC curve
I wanted to find the ability of a numeric continuous variable to predict mortality (dead/alive), and what cut off value to take in the continuous variable.
I wanted to construct an ROC curve and find ...
0
votes
0answers
11 views
ROC for time-varying covariates in ddhazard model?
i want to draw ROC for ddhazard model. I have tried risksetROC package. But i am unable to get predictors for my model. here is the code:
OBTAIN A LINEAR PREDICTOR
predict1<- predict(ddfit1,type =...
1
vote
1answer
41 views
How to calculate and plot cross-validated ROC?
I am doing K-fold cross validation and I want to plot an averaged ROC curve in MATLAB. However currently I can only plot K ROC curves in one plot but without knowing the algorithm of averaged ROC ...
3
votes
0answers
44 views
Power and sample size calculations for unpaired ROC curves
I would like to perform power and sample size calculations for comparison of unpaired receiver-operating characteristic (ROC) curves. I have tried using the power.roc.test function from pROC package ...
-1
votes
1answer
26 views
R-Caret : Not-meaningful class probabilities and AUC value
I am very new to ML therefore my question might be primitive. I am working on a binary-class problem. The response (target) variable is occurrence : a factor ...
0
votes
0answers
29 views
Finding optimal cutoff point [duplicate]
.Hello,everyone.
I am studying the influence of one biomarker on multiple disease characteristics, and I would like to calculate its cutoff point. I created univariate ROC curves to investigate the ...
0
votes
1answer
25 views
Possible reasons for AUC=1 (from a fitted glm model)?
I am running high-throughput microarray data (methylation array), and after running univariate, lasso and cross-validation lasso analyses, I was able to come down to a list of 15 probes (predictors).
...
1
vote
1answer
59 views
ROC Curve Horizontal Line
I'm trying to conceptually understand 'why' the horizontal gray line in ROC curve below would be considered random.
I know the basics of the ROC in that the farther the curved black line is from the ...
0
votes
1answer
36 views
Calculate AUC using predicted values and labels from a 5 fold classification?
I have a classifier for a binary problem. That has outputs between 0 and 1 for predictions for the two class A or B (for example sunny, not sunny). The classifier has ran on 5 unique folds of the data ...
0
votes
1answer
53 views
How to decide threshold values for AUC?
With respect to the pROC package for R(https://rdrr.io/cran/pROC/man/ggroc.html).
...
2
votes
0answers
20 views
Prove that AUROC is an improper scoring rule [duplicate]
It has been stated in many places that AUROC is an improper scoring rule.But I haven't seen anyone proving it. Does someone have a working example that shows that maximizing AUROC actually moves away ...
0
votes
1answer
47 views
1
vote
0answers
29 views
When do ROC-Curves intersect?
In books I see ROC curves like this:
So I always think the model with the highest AUC is better at any threshold. Is this true in general?
When applying usual classification models, can there be ...
2
votes
1answer
166 views
glmStepAIC model is doing better that other models
I am training a model on an imbalanced dataset (about 5-20% of positive class) and trying out different algorithms in R using caret package.
I have 57 predictors and around 2000-3000 observations in ...
0
votes
2answers
281 views
Calculate AUC using sensitivity and specificity values only
How to calculate AUC, if I have values of sensitivity and specificity for various threshold cutoffs?
I have sensitivity and specificity values for 100 thresholds.
...
0
votes
0answers
25 views
3D ROC surface from multinomial logistic regression with more than one predictor
How do you construct a 3D ROC surface in R from a multinomial logistic regression? I have been able to form a 3D ROC surface for the individual models within a multinomial logistic regression, but not ...
0
votes
2answers
62 views
Compare two Roc curves with same Auc
I have 2 churn prediction model.
Both provided very similar AUC values for Roc but with different shape.
How should I assess which to choose based on that fact?
3
votes
2answers
165 views
PR AUC < 50% with ROC AUC > 90% - model good or bad?
I understand for highly imbalanced dataset - we need to look for precision-recall vs ROC AUC to better judge the model.
My question is what is the range for PR AUC below which the model is bad? My ...
1
vote
0answers
18 views
AUC-like measure for multiple simultaneous classification tasks?
I know that given an ordered set of binary labels, and equally-sized ordered set of scalar predictions, we can quantify how cleanly the predictions separate the labels into clean buckets of 0's and 1'...
2
votes
1answer
54 views
ROC-style curves for calculating sample size, power, alpha, and effect size
I found an awesome R package called pwr that does all sorts of calculations about sample sizes, power, effect sizes, and so on, and I've been playing.
I have a ...
0
votes
0answers
30 views
ROC curve for a perfect classifier
I have trained a DNN classifier on a balanced dataset which outputs softmax values $p_1$ and $p_0$ corresponding to probabilities of positive (label 1) and negative (label 0) classes, respectively. I ...
2
votes
1answer
126 views
Generate synthetic data given AUC
I'm experimenting with ROC-AUC for binary classification problems. I want to generate synthetic data for a given AUC score. The ...
0
votes
2answers
79 views
Statistical evidence that the AUC was not overfitted to the model. With N=119, C-stat = 0.81 seems optimistic. Optimism-adjusted?
My data have 119 cases and we did ROC for x (continuous variable) to predict postoperative y (categorical variable) available here, we got a comment from a reviewer asking:
Please provide ...
0
votes
0answers
14 views
Statistical evidence that the AUC was not overfitted to the model. With N=119, C-stat = 0.81 seems optimistic. Optimism-adjusted? [duplicate]
My data has 119 case and We did ROC for x (continuous variable ) to predict postoperative y (categorical variable) available here, we got a comment from a reviewer asking "Please provide statistical ...
0
votes
1answer
24 views
Area under the ROC curve accuracy [duplicate]
Why is the area under the ROC curve better than raw accuracy as an out-of- sample evaluation metric?
2
votes
1answer
55 views
SVM: achieving ROC curve by varying misclassification costs
Assume that we have SVM model for binary classification with objective function as follows:
$$
min(\frac{1}{2}\omega\cdot\omega +C^{+}\sum_{i|y_{i}=+1}^{n}\xi_{i}\quad+C^{-}\sum_{i|y_{i}=-1}^{n}\xi_{i}...
1
vote
3answers
72 views
How to graph the difference between similar ROC curves
I have a model and the ROC curve it produces, I modified the model and it produced a second ROC curve that is very similar in shape to the first.
If I graph these ROC curves on the same plot, then it ...
0
votes
0answers
25 views
Which is best method to find optimal cut-off for ROC for continuous data?
Besides youden index can anyone tell me which is the best method to find cut- off point for a ROC CURVE for continuous measurements.
