# Questions tagged [roc]

Receiver Operating Characteristic, also known as the ROC curve.

776 questions
Filter by
Sorted by
Tagged with
10 views

### Can I use ROC/AUC to compare binomial GLMM models?

Is there any reason not to use ROC for evaluating the effectiveness of a binomial GLMM model with logit link? In my specific case, I have six models (one null model with two random effects, five other ...
25 views

### Comparing AUCs for two ROC curves

I have a data analysis puzzle involving ROC curves that I hoped you could help me with. One of my research projects involves exploring how to use crowds to do idea filtering (i.e. to distinguish good ...
15 views

### Calculating Area Under the Curve Receiver Operating Characteristic AUC ROC [duplicate]

So on wikipedia here is the definition for area under the curve of the receiver operating characteristic. Can someone give a simple example of calculating this given some predictive results in a ...
1k views

### Why is ROC insensitive to class distributions?

I am confused over why ROC is invariance under class distribution described in the paper An Introduction to ROC analysis. I cannot understand the example on why the proportion of positive to negative ...
9 views

### How can i find threshold value via 3 way ROC analysis and plot? [closed]

How can i find threshold value via 3 way ROC analysis and plot? I have a med data and need to find cut point. Could you please help me?
22 views

### How to draw a ROC curve with multiple models?

Here is the condition: A, B and C are three different types of categories. A is negative, B and C are both positive. I trained two models. Model 1 is to recognize only B (set A and C as negative, B as ...
45 views

### ROC curves with a sharp elbow/V shape

Can someone tell me why my ROC curve looks like this? roc(data,response,predictor,plot=T) Area under the curve: 0.899 ...
289 views

### How are True Negative and False Negative converted into True Positive and False Positive in ROC curve?

I found the ROC explanation at this link. It states that the ROC curve is TP vs FP. After the ...
33 views

### How to calculate drug enrichment curves using AUC?

I have data on various drugs and p values describing how strongly these drugs are associated with a given disease (e.g., type 2 diabetes; p values calculated with gene set analysis). I want to ...
29 views

### How to calculate p-value to compare AUPRC for two models?

I've built two deep learning models and was looking for a way to compare the two models in terms of the area under the precision recall curve terms. I know that the DeLong method is recommended for ...
32 views

### About ROC curve in segmentation model

I know how to draw ROC curves about classification model for a one class. And I know how to plot ROC curves about classification model for many classes. But is there a way to plot ROC curves for a ...
15 views

### ROC Curve for Fooling a Classifier

Suppose I have some classes, $A$ and $B$. I then perform some function $f(A) = A'$ and $g(A) = A''$, designed to be as close to $B$ as possible. I would like to perform a fair test to measure the ...
21 views

### Can I use the ROC curve to compare two distributions?

The ROC curve for two distributions $F$ and $G$ can be defined as $$\mbox{ROC}(u) = {F}(G^{-1}(u)),$$ for $u \in (0,1)$. So, if $F=G$, then $\mbox{ROC}(u) = u$. Can I use this property to compare the ...
7 views

### Comparing area under ROC for validation and blind set

I used this tool http://vassarstats.net/roc_comp.html to calculate the significance value between the area under the ROC of my validation set and area under the ROC of my blind set. Both were ...
27 views

### Comparing test and validation ROC curves statistically

So I have two sets of data my validation set and my test set. I have a neural network model that was tuned on the validation set and then finally tested on the test set and I got similar results in ...
9 views

### Why is c-index useful when it is derived from time-backward sensitivity and specificity?

Concordance index with a binary outcome is equivalent to area under the receiver-operator curve. Thus, c-index is based on sensitivity and specificity. As Frank Harrell has pointed out, both on Cross ...
74 views

### When to use smoothing of the predictor in a time dependent incident dynamic ROC analysis, and is there any arguments not to use smoothing?

In the article “Estimation of incident dynamic AUC in practice”, Geloven et al. 2020 (https://www.sciencedirect.com/science/article/pii/S0167947320301869#b10) they describe: “In recent years, several ...
24 views

### Given specificity and sensitivity, how do we get ROC value for KNN [R]

I trained a KNN model with k fold cross validation, and I got the following results. I know how specificity and sensitivity are calculated, but the ROC value I don't know. How is that estimated? For ...
27 views

### Comparing ROC AUPRC scores in case of different baselines

I have some imbalanced data for binary classification, which I have preprocessed in 2 different ways. That led to having a different number of observations and pos/neg ratio. Then I trained the same ...
44 views

### Evaluation metric for time-series anomaly detection

My dataset is time-series sensor data and anomaly ratio is between 5% and 6% 1. For time-series anomaly detection evaluation, which one is better, precision/recall/F1 or ROC-AUC ? When empirically ...
7 views

### Practical calculation of EER (equal error rate) for biometric tasks

I am recently experimenting with the speaker recognition task. So, EER is calculated for a threshold FAR = FRR. Now, my question is how can I calculate this given I ...
94 views

### Predictive discrimination of a single parameter

I the clinical medical imaging literature I often read that certain imaging parameters "predict" certain outcomes better than other imaging parameters. These conclusions are often drawn from ...
382 views

### Why isn't the ROC curve naturally plotted in 3D? [duplicate]

Something that really confuses me with how ROC plots are generated is that, according to Wikipedia: The ROC curve is created by plotting the true positive rate (TPR) against the false positive rate (...
20 views

### True Positive Rate (TPR), and False Positive Rate (FPR) to evaluate binary classifier in the anomaly detection domain

I am confused with the the binary classifier evaluation criteria in anomaly detection domain for highly imbalanced data. I see that some authors claimed that their model performed excellent to predict ...
80 views

### Why is the ROC curve two-dimensional instead of three-dimensional?

When I understand it correctly, a ROC curve composes of a lot of ratios of TPR and FPR. Each such ratio is gained, e.g., by scanning/investigating a parameter where the ratio is the result of the used ...
19 views

### Why ROC curves gives different ranking of classes than confsuion matrix?

I am applying logistic regression in a imbalance class datset which has 16 classes. When I see the confusion matrix I found that best predicted class is 5 having the score of 0.97 then class 15 having ...
60 views

### ROC curve and optimal probability threshold (cut off) [duplicate]

When we draw a ROC curve, how can we determine the optimal threshold? I think, the point that is closest to (0,1) vector is the best threshold. However, if we cannot calculate the distance, how can we ...
37 views

### ROC curve from an array of Confusion Matrices (true positive rates and false positive rates)

How can we create an ROC curve from an array of Confusion Matrices (true positive rates and false positive rates)?
85 views

### Calculate AUC based on TPR and FPR

I have two equations for TPR and FPR (based on the threshold t), for example: TPR = (1-t)^2 FPR = (1-t)^0.2 How can I calculate ...
12 views

### Choosing the Best Tuning Parameter Considering the Cutoff Point in Classification

Suppose that we want to predict a binary response "y" using "LASSO". We may first want to select the tuning parameter lambda (shrinkage parameter) using cross-validation. Here, the ...
18 views

### SVM reports different AUC if binary labels 0 and 1) are swapped

I am using a SVM classifier on an highly imbalanced binary data-set (about 10:1 ratio of the majority class to the minority class). I assign the majority class a label of 1 and the minority class to 0....
15 views

### Demonstrating that proposed cutoff score is non optimal

I have a small data set with a test score and classification (positive or negative diagnosis). There is already clinical evidence that the cut point suggested by the test manufacturer produces too ...
23 views

### ROC curve with repeated thresholds?

I am reading this paper, and Figure 4 looks as follows: As I understand ROC curves the numbers on the curve are typically the thresholds for binary classifications. Hence how comes that here we have ...
66 views

### How does time specific hazard ratios relate to time dependent incident dynamic ROC AUC?

I have a question related to time specific hazard and time dependent incident dynamic ROC analysis. I am currently working on some analysis where I evaluate a prognostic model (Cox model) on some ...
38 views

### Determining the winner model from ROC curve

I have a ROC curve for a specific hyperparameter tuning setting for a decision tree. The candidate values for which I plot are 0.1, 0.01, 0.001, 0.0001. I want to determine (visually) which model has ...
4 views

### Analogue of Youden's index if both variables are continuous?

Let's say that we have one binary variable and one continuous variable which we want to dichotomize in a way that we find a threshold which will make the continuous variable as similar to the binary ...
25 views

### How to take account of the variable number of contributions of statistical individuals in a ROC curve?

Consider data of patients in an hospital. For each stay, we have a score from 0 to 100 and the presence or absence of a diagnosis. We want to draw a ROC curve where a threshold on the score will ...
25 views

### Regarding ROC curve of good classifier, why TPR and FPR both increase? [duplicate]

As title, I haven't found a post explain about this, i.e. In the following diagram, for a point on a "better" ROC curve and I move that point along the curve toward the top-right corner, why ...
77 views

### What is the reason for the difference in AUC from probabilities vs AUC from final prediction

I have a binary classification model. The target variable is my_test_data\$target_variable and has values 'y' or 'n'. ...
22 views

### I am participating in my first hackathon and I have a doubt regarding public and private leaderboard

In the competition that I am participating 40 percentage of the data is taken for public leaderboard and 60 percentage is reserved for private leaderboard. Question 1 On public leaderboard I'm getting ...
33 views

### Interpreting classification metrics for multiclass imbalance

I am at the point of reporting my results in a research article conducted. The dataset is highly imbalanced with class 1 and ...
64 views

### Why do we look at multiple thresholds in AUC

For AUC ROC, why do we have a graph of multiple thresholds when in the end, we will only use one of those thresholds (so why not just choose the threshold and compare that one value across models)?
48 views

### Micro average for AUC/ROC

The yellowbrick documentation has an example of AUC/ROC here. It seems odd to me the micro average would have an AUC much bigger than the individual classes or macro average. Is there some reason the ...
32 views

### Why is ROC a ranking metric and how do we understand it as a c-statistic?

I have been stuck on this topic for quite a long time. I still do not understand why AUROC is a ranking metric. In particular, what does "ranking" mean in AUROC? When I tried to code ROC out ...
12 views

### Can you do hyperparameter tuning using a PR curve? Moreover, would this still be considered a "PR curve"?

I am creating a graphical representation of PR results across various hyperparameter changes in an imbalanced dataset (model used was an SVM). I'm wondering if one would still consider this a "PR ...
15 views

### Why does using probabilities result in higher ROC_AUC scores?

in the most recent playground competition on kaggle (https://www.kaggle.com/c/tabular-playground-series-mar-2021/overview/evaluation) we once again have an evaluation via the area under the roc curve. ...
20 views

### imbalanced classes: ROC_AUC vs Precision_Recall AUC

I am dealing with a highly imbalanced classes problem. Accuracy is of course not a good performance metric in such cases, So I want to calculate either ROC AUC sore ...
48 views

### Why is the diagonal line in ROC space called the random guess?

all. I'm a student who studies machine learning. When I found the wiki over the ROC(Reciever Operation Characteristic), I had one question. According to the wiki, ROC space has a diagonal line and ...