Questions tagged [roc]
Receiver Operating Characteristic, also known as the ROC curve.
691
questions
0
votes
0answers
12 views
threshold coded ROC curve interpretaion
I'm trying to figure out ROC curve better and must be kind of dumb but i don't know what are the thresholds of right side of image saying!
On this graph, the y-axis is true positive rate, and the x-...
0
votes
0answers
8 views
How to remove the effect of age on the ROC curve?
I have data set, and the outcome variable is the prognosis (survival or death). I want to evaluate the prognosis of the patient by the SOFA scores, but the problem now is that the age of this data set ...
0
votes
1answer
36 views
Threshold ROC vs. Cut-Off in confusion matrix (binary regression)
I am trying to understand the link between the threshold in ROC-analysis and the threshold defined in classification table. Criterion is binary with 0 or 1.
1) Someone can determine a confusion ...
0
votes
0answers
7 views
Explaining a Conformal Prediction (vs NonConformal)
I'm trying to understand the definition of a conformal prediction and possibly the difference with a non-conformal prediction. What I get as a general idea is that the conformal prediction will be ...
0
votes
0answers
12 views
Cumulative Accuracy Profile(CAP) Curve vs Receiver Operator Characteristic (ROC) Curve
While going through several metrics such as Accuracy and Recall to measure the performance of a Machine Learning model, I found in some articles that ROC Curve and CAP Curve are great for ...
0
votes
0answers
10 views
ROC for a family of repeated experiments
Suppose I have a continuous covariate like age, and I want to use it for RNA expression which means that I have to fit a large number of linear models, one per gene, that predict the expression. ...
0
votes
1answer
33 views
How do I display my output comparing the effect of variables on classification of disease from a random forest analysis in R using an AUROC?
I am using a random forest classification to compare how the classification of disease improves when combining metabolites with two other measures ( visceral fat and CRP-1 levels) to see if adding the ...
2
votes
1answer
29 views
Plot ROC curve in R with the PRROC package
After many hours of research, trial and error and frustration I hope someone here will be able to guide me in the right direction. I am fairly new to R and statistics and can not wrap my hand about ...
1
vote
0answers
22 views
Calculating three average precisions and a single value for ROC from raw predicted class outputs
I'm not a statistician or mathematician so I apologize if I use any terms incorrectly. Please do point out any errors in my use of terminology.
The four values I need are the equivalent of Weka's ROC ...
1
vote
1answer
52 views
Why can't my ROC reach a low PPV?
I am creating a couple of models (RF, SVM, LR) and I want to evaluate all of them on a certain PPV (0.7).
This question and this question helped me write my code:
...
0
votes
1answer
24 views
When is a dataset ātoo imbalancedā for AUC ROC and PR is preferred?
Iāve read that precision-recall (PR) curves are preferred over AUC-ROC curves when a dataset is imbalanced as thereās more of a focus on the modelās performance in correctly identifying the minority/...
0
votes
1answer
18 views
how to increase the output values without increasing datasets?
I am using MatLab 2017a. I have dataset of 12 video. I have to plot ROC curve of this dataset. For this purpose, I have used perfcurve command.
...
0
votes
0answers
15 views
retrieve best cutoff of ROCR and compare?
I am new to ROCR curves and I wanted to use them to evaluate some methods:
I am using the same code as in āintroduction to statistical learningā, that is:
...
1
vote
1answer
27 views
Normalise different thresholds for binary prediction
I'm working in a module that outputs the risk of an event happening i.e. risk of a crime happening depending on the district of the city. What I've done is to calculate for each district a binary ...
0
votes
0answers
11 views
How to calculate ROC curve at a threshold value
I have to calculate the ROC of 12 different videos. I have taken a code from this LINK. Following is there code of calculating TPR and FPR:
...
2
votes
1answer
39 views
Area under the curve score training/validation set
Lets say I have a very basic, binary classification problem and I use logistic regression.
The logistic regression will give me a score (not a classification yet), between 0 and 1.
I can use sklearn'...
4
votes
1answer
108 views
Creation of ROC curve of logistic regression classifier with rejection
For a logistic regression classifier, I create a roc curve by variation of the threshold on the output probability.
Question: can I create an additional ROC curve with 5% rejection rate based on the ...
0
votes
0answers
13 views
Can a classifier's ROC AUC be compared to a person's accuracy at correctly ranking a positive, negative pair?
I've read that ROC AUC is equivalent to the probability of correctly ranking a positive, negative pair. If a human were to attempt to correctly rank a bunch of positive, negative pairs, could their ...
1
vote
0answers
7 views
How to draw Receiver Operating Characteristics (ROC) of 34 video sequences using MatLab
I am working on video processing using MatLab R2017a. I have datasets containing 34 video sequences. I have to calculate the AUC of this datasets. As I know AUC is dependent upon the ROC. And ROC is a ...
0
votes
0answers
20 views
PR vs ROC AUC curve
I'm working on a model which decides whether a bank transaction is relevant for an acceptant to review or not. This process is now done by hand when someone applies for a loan. Bank transactions are ...
3
votes
1answer
30 views
Significance testing for comparing ROC areas
I have been analyzing the accuracy of 3 prognostication scores in predicting a certain binary outcome using ROC curves and significance testing for differences in AUCs between the curves (a figure of ...
3
votes
1answer
107 views
Mann Whitney U and ROC AUC relationship
I've been learning about the relationship between Mann-Whitney U.
Supposedly, the area under the ROC curve should be
U/(n0 * n1), where U is the Mann-Whitney ...
1
vote
0answers
23 views
Generating confidence interval for difference between 2 AUCs
I am trying to get a confidence interval for the difference between 2 AUCs for paired ROC curves. I've been reading the paper pROC: an open-source package for R and S+ to analyze and compare ROC ...
0
votes
1answer
21 views
How to determine the best model based on ROC curves
I have created four models of Logistic Regression (binary classification) and have developed the ROC curves of those 4 models.
However, I am not able to understand how to scientifically determine the ...
1
vote
0answers
24 views
Relationship between AUC of ROC curve and AUC of PR (precision-recall) curve
I know that both the ROC curve and the PR curve can be used to evaluate the performance of a binary classification prediction model, and PR curve is preferred in the case of imbalanced class ...
3
votes
1answer
43 views
Slope of ROC increases
Hi my ROC curve seems weird because in most of the cases, the slope of ROC curve should decrease.
Can anyone help to interpret this? Huge thanks!
0
votes
0answers
14 views
Why would my additional information harm my prediction score but improve ROC and F-1?
I'm trying to predict the primary crime type on a given location using the Chicago crime dataset.
Stripping out all the provided features to just:
Location Description Encoded (The location ...
1
vote
0answers
29 views
How to interpret an AUC value of 0.55
I have a classifier with an AUC value of 0.55. I understand that an AUC value of 0.5 is essentially a random classifier and not predictive at all. So I am wondering if an AUC of 0.55 is ...
1
vote
0answers
17 views
OneR and area under the ROC curve
In the book Data Mining, author Ian Witten says that it does not make sense to calculate the area under the ROC curve when applying OneR because it finds only one point (sensitivity, specificity) ...
0
votes
0answers
12 views
How does roc_auc_score() calculates score using only one set of y_true and y_predicted? [duplicate]
ROC curve is generated using different thresholds in classification model hence getting different TPR(True Positive Rate) and FPR(False Positive Rate) for every prediction. But we only pass one set of ...
0
votes
1answer
22 views
Why is an ROC curve TPR (Y) against FPR (X)?
I am trying to thoroughly understand the ROC Curve and I was wondering why is an ROC Curve always (seemingly) TPR against FPR? I have had discussions with others about this matter and I cannot think ...
0
votes
0answers
8 views
Problems while implementing cross-validation and ROC-curve
I am trying to implement my understanding of the cross-validation and ROC-curve. However, after the computations have been done I can see that the precision is 1 for every parameter and every ...
0
votes
1answer
31 views
Finding optimal point in roc curve giving weighs to true positives and false negatives
I have a binary classification model whose ROC curve looks like the one below. The black point is the optimal probability threshold to use by calculating the geometric mean. However, that's a pretty ...
0
votes
0answers
7 views
How to understand the mathematics of ROC/AUC in Wikipedia [duplicate]
Could someone help explain what happened in the red box? I'm struggling to figure out where the indicator function comes from in this calculation. Thank you~ wiki link
0
votes
0answers
10 views
What does using the probabilities of positive class mean for area under the curve?
In many of the scikit models, there is a predict_proba method that returns a NxM matrix, ...
0
votes
0answers
10 views
How to bulid a confusion matrix for videos from single frame classifications?
assume I have a sequence of individual inputs that make up one piece of data. For instance, a video is comprised of a sequence of image frames.
Let's say I have built a classifier for certain things ...
2
votes
1answer
31 views
Adding predictors in ROC curves and how does it affect AUC?
I have a general question about ROC curves and how adding predictors affect AUC values.
Let's say I have a model that contains only predictor A and produces an AUC of 0.6.
I then add into the model a ...
0
votes
1answer
40 views
precision recall AUC or ROC AUC question
I'm working on a project where the 'in the wild' prevalence is a significant imbalance (e.g. minority 4%). However, I was able to collect data in a balanced manner, i.e. 4,000 samples of minority and ...
0
votes
0answers
29 views
How to treat large chunks of missing data when ROC-AUC scoring is used
I have a dataset with a majority of features for about one-third of both train and test data are missing. E.g. I have values A, B, and C for 66% of my data and only C's for the rest of the data.
The ...
2
votes
0answers
66 views
ROC Convex Hull for Model Selection
The area under the convex hull of a roc curve is by construction always "better" than its area under curve. Some curves might see more of an increase in reported auc than others. Is it a viable ...
1
vote
1answer
36 views
ROC Curve AUC for Hypothesis Testing Sensitivity (Power) vs Specificity ($1-\alpha$)
Let's set up a hypothesis test of $H_0: \theta=\theta_0$ versus $H_1: \theta\ne\theta_0$, and let's say that I have two techniques to assess this (say equal-variance t-test versus unequal-variance ...
1
vote
0answers
101 views
The usage of TOC curves?
In machine learning literature ROC curves are common performance measures. Quite recently published and not as popular (50+ citatations) are TOC curves as proposed by Pontius Jr, Robert Gilmore and Si,...
0
votes
1answer
50 views
How to “draw” a ROC curve [duplicate]
I have read this question but it doesn't have any well-explained answer for this case: I understand the ROC curve in overall, but I'm looking for a step-to-step explanation in order to understand how ...
2
votes
0answers
27 views
Odd results of AUC and McFadden R2 on insure tech article
I was reading this article entitled: Usage-Based Vehicle Insurance: Driving Style Factors of Accident Probability and Severity (Korishchenko et all., 2019) [1], and watching the results section, I saw ...
0
votes
0answers
11 views
AUROC and Equality in Scores
The task at hand is to calculate the empirical AUROC for scores assigned to $N + M$ samples from two classes A and B.
Most scores lie within the interval (0,1) and are pairwise different, but some ...
0
votes
0answers
15 views
Is it normal to have a high AUC in train test
I have Random Forest classification model which is already tunned using k-folds cross validation, when I train the model in the train set, the auc gives me 0.97, in test set is 0.75.
Is it normal to ...
0
votes
0answers
18 views
Connection between prob output LogisticReg/SVM and ROC
I have the following ROC generated using LPOCV and Logistic regression or SVM (l2 norm). Now, let's say I have a test set containing 10 patients and I get that the probabilities of those patients to ...
1
vote
1answer
19 views
Confidence band for the ROC curve
How would you construct a confidence band for the ROC curve?
I do not mean a confidence interval for the AUC or for other parameters that can be derived from the ROC curve, but a confidence band for ...
0
votes
0answers
17 views
Feature subset selection or find classifier threshold first for logistic regression?
I have a baseline model with 137 variables and 300 observations, and I am trying to solve a classification problem with a logistic model. I understand that the base model won't have much value.
In ...
1
vote
1answer
47 views
AUC ROC and probabilistic interpretation
I can't solve the problem about the AUC ROC metric. Problem condition: on the answers (estimates) of the algorithm, objects of class 0 are distributed uniformly on the segment [0, 2/3], and answers of ...
