Questions tagged [roc]
Receiver Operating Characteristic, also known as the ROC curve.
776
questions
0
votes
0answers
10 views
Can I use ROC/AUC to compare binomial GLMM models?
Is there any reason not to use ROC for evaluating the effectiveness of a binomial GLMM model with logit link? In my specific case, I have six models (one null model with two random effects, five other ...
1
vote
0answers
25 views
Comparing AUCs for two ROC curves
I have a data analysis puzzle involving ROC curves that I hoped you could help me with.
One of my research projects involves exploring how to use crowds to do idea filtering (i.e. to distinguish good ...
0
votes
0answers
15 views
Calculating Area Under the Curve Receiver Operating Characteristic AUC ROC [duplicate]
So on wikipedia here is the definition for area under the curve of the receiver operating characteristic.
Can someone give a simple example of calculating this given some predictive results in a ...
11
votes
4answers
1k views
Why is ROC insensitive to class distributions?
I am confused over why ROC is invariance under class distribution described in the paper An Introduction to ROC analysis. I cannot understand the example on why the proportion of positive to negative ...
0
votes
0answers
9 views
How can i find threshold value via 3 way ROC analysis and plot? [closed]
How can i find threshold value via 3 way ROC analysis and plot?
I have a med data and need to find cut point. Could you please help me?
0
votes
0answers
22 views
How to draw a ROC curve with multiple models?
Here is the condition: A, B and C are three different types of categories. A is negative, B and C are both positive. I trained two models. Model 1 is to recognize only B (set A and C as negative, B as ...
0
votes
0answers
45 views
ROC curves with a sharp elbow/V shape
Can someone tell me why my ROC curve looks like this?
roc(data,response,predictor,plot=T)
Area under the curve: 0.899
...
5
votes
2answers
289 views
How are True Negative and False Negative converted into True Positive and False Positive in ROC curve?
I found the ROC explanation at this link. It states that the ROC curve is TP vs FP.
After the ...
1
vote
1answer
33 views
How to calculate drug enrichment curves using AUC?
I have data on various drugs and p values describing how strongly these drugs are associated with a given disease (e.g., type 2 diabetes; p values calculated with gene set analysis). I want to ...
1
vote
0answers
29 views
How to calculate p-value to compare AUPRC for two models?
I've built two deep learning models and was looking for a way to compare the two models in terms of the area under the precision recall curve terms. I know that the DeLong method is recommended for ...
0
votes
1answer
32 views
About ROC curve in segmentation model
I know how to draw ROC curves about classification model for a one class. And I know how to plot ROC curves about classification model for many classes. But is there a way to plot ROC curves for a ...
0
votes
0answers
15 views
ROC Curve for Fooling a Classifier
Suppose I have some classes, $A$ and $B$. I then perform some function $f(A) = A'$ and $g(A) = A''$, designed to be as close to $B$ as possible.
I would like to perform a fair test to measure the ...
0
votes
1answer
21 views
Can I use the ROC curve to compare two distributions?
The ROC curve for two distributions $F$ and $G$ can be defined as
$$\mbox{ROC}(u) = {F}(G^{-1}(u)),$$
for $u \in (0,1)$. So, if $F=G$, then $\mbox{ROC}(u) = u$. Can I use this property to compare the ...
0
votes
0answers
7 views
Comparing area under ROC for validation and blind set
I used this tool http://vassarstats.net/roc_comp.html to calculate the significance value between the area under the ROC of my validation set and area under the ROC of my blind set.
Both were ...
0
votes
1answer
27 views
Comparing test and validation ROC curves statistically
So I have two sets of data my validation set and my test set. I have a neural network model that was tuned on the validation set and then finally tested on the test set and I got similar results in ...
0
votes
0answers
9 views
Why is c-index useful when it is derived from time-backward sensitivity and specificity?
Concordance index with a binary outcome is equivalent to area under the receiver-operator curve. Thus, c-index is based on sensitivity and specificity.
As Frank Harrell has pointed out, both on Cross ...
4
votes
1answer
74 views
When to use smoothing of the predictor in a time dependent incident dynamic ROC analysis, and is there any arguments not to use smoothing?
In the article “Estimation of incident dynamic AUC in practice”, Geloven et al. 2020 (https://www.sciencedirect.com/science/article/pii/S0167947320301869#b10) they describe:
“In recent years, several ...
0
votes
0answers
24 views
Given specificity and sensitivity, how do we get ROC value for KNN [R]
I trained a KNN model with k fold cross validation, and I got the following results. I know how specificity and sensitivity are calculated, but the ROC value I don't know. How is that estimated?
For ...
0
votes
0answers
27 views
Comparing ROC AUPRC scores in case of different baselines
I have some imbalanced data for binary classification, which I have preprocessed in 2 different ways. That led to having a different number of observations and pos/neg ratio. Then I trained the same ...
1
vote
1answer
44 views
Evaluation metric for time-series anomaly detection
My dataset is time-series sensor data and anomaly ratio is between 5% and 6%
1.
For time-series anomaly detection evaluation, which one is better, precision/recall/F1 or ROC-AUC ?
When empirically ...
0
votes
0answers
7 views
Practical calculation of EER (equal error rate) for biometric tasks
I am recently experimenting with the speaker recognition task.
So, EER is calculated for a threshold FAR = FRR.
Now, my question is how can I calculate this given I ...
1
vote
1answer
94 views
Predictive discrimination of a single parameter
I the clinical medical imaging literature I often read that certain imaging parameters "predict" certain outcomes better than other imaging parameters.
These conclusions are often drawn from ...
4
votes
3answers
382 views
Why isn't the ROC curve naturally plotted in 3D? [duplicate]
Something that really confuses me with how ROC plots are generated is that, according to Wikipedia:
The ROC curve is created by plotting the true positive rate (TPR)
against the false positive rate (...
0
votes
0answers
20 views
True Positive Rate (TPR), and False Positive Rate (FPR) to evaluate binary classifier in the anomaly detection domain
I am confused with the the binary classifier evaluation criteria in anomaly detection domain for highly imbalanced data. I see that some authors claimed that their model performed excellent to predict ...
4
votes
1answer
80 views
Why is the ROC curve two-dimensional instead of three-dimensional?
When I understand it correctly, a ROC curve composes of a lot of ratios of TPR and FPR. Each such ratio is gained, e.g., by scanning/investigating a parameter where the ratio is the result of the used ...
1
vote
0answers
19 views
Why ROC curves gives different ranking of classes than confsuion matrix?
I am applying logistic regression in a imbalance class datset which has 16 classes. When I see the confusion matrix I found that best predicted class is 5 having the score of 0.97 then class 15 having ...
1
vote
1answer
60 views
ROC curve and optimal probability threshold (cut off) [duplicate]
When we draw a ROC curve, how can we determine the optimal threshold?
I think, the point that is closest to (0,1) vector is the best threshold.
However, if we cannot calculate the distance, how can we ...
2
votes
2answers
37 views
ROC curve from an array of Confusion Matrices (true positive rates and false positive rates)
How can we create an ROC curve from an array of Confusion Matrices (true positive rates and false positive rates)?
1
vote
1answer
85 views
Calculate AUC based on TPR and FPR
I have two equations for TPR and FPR (based on the threshold t), for example:
TPR = (1-t)^2
FPR = (1-t)^0.2
How can I calculate ...
0
votes
0answers
12 views
Choosing the Best Tuning Parameter Considering the Cutoff Point in Classification
Suppose that we want to predict a binary response "y" using "LASSO".
We may first want to select the tuning parameter lambda (shrinkage parameter) using cross-validation.
Here, the ...
0
votes
1answer
18 views
SVM reports different AUC if binary labels 0 and 1) are swapped
I am using a SVM classifier on an highly imbalanced binary data-set (about 10:1 ratio of the majority class to the minority class). I assign the majority class a label of 1 and the minority class to 0....
0
votes
0answers
15 views
Demonstrating that proposed cutoff score is non optimal
I have a small data set with a test score and classification (positive or negative diagnosis). There is already clinical evidence that the cut point suggested by the test manufacturer produces too ...
3
votes
1answer
23 views
ROC curve with repeated thresholds?
I am reading this paper, and Figure 4 looks as follows:
As I understand ROC curves the numbers on the curve are typically the thresholds for binary classifications. Hence how comes that here we have ...
3
votes
1answer
66 views
How does time specific hazard ratios relate to time dependent incident dynamic ROC AUC?
I have a question related to time specific hazard and time dependent incident dynamic ROC analysis.
I am currently working on some analysis where I evaluate a prognostic model (Cox model) on some ...
2
votes
1answer
38 views
Determining the winner model from ROC curve
I have a ROC curve for a specific hyperparameter tuning setting for a decision tree. The candidate values for which I plot are 0.1, 0.01, 0.001, 0.0001. I want to determine (visually) which model has ...
0
votes
0answers
4 views
Analogue of Youden's index if both variables are continuous?
Let's say that we have one binary variable and one continuous variable which we want to dichotomize in a way that we find a threshold which will make the continuous variable as similar to the binary ...
0
votes
0answers
25 views
How to take account of the variable number of contributions of statistical individuals in a ROC curve?
Consider data of patients in an hospital. For each stay, we have a score from 0 to 100 and the presence or absence of a diagnosis.
We want to draw a ROC curve where a threshold on the score will ...
0
votes
1answer
25 views
Regarding ROC curve of good classifier, why TPR and FPR both increase? [duplicate]
As title, I haven't found a post explain about this, i.e. In the following diagram, for a point on a "better" ROC curve and I move that point along the curve toward the top-right corner, why ...
2
votes
1answer
77 views
What is the reason for the difference in AUC from probabilities vs AUC from final prediction
I have a binary classification model. The target variable is my_test_data$target_variable and has values 'y' or 'n'. ...
1
vote
0answers
22 views
I am participating in my first hackathon and I have a doubt regarding public and private leaderboard
In the competition that I am participating 40 percentage of the data is taken for public leaderboard and 60 percentage is reserved for private leaderboard.
Question 1
On public leaderboard I'm getting ...
1
vote
1answer
33 views
Interpreting classification metrics for multiclass imbalance
I am at the point of reporting my results in a research article conducted. The dataset is highly imbalanced with class 1 and ...
2
votes
1answer
64 views
Why do we look at multiple thresholds in AUC
For AUC ROC, why do we have a graph of multiple thresholds when in the end, we will only use one of those thresholds (so why not just choose the threshold and compare that one value across models)?
0
votes
0answers
48 views
Micro average for AUC/ROC
The yellowbrick documentation has an example of AUC/ROC here. It seems odd to me the micro average would have an AUC much bigger than the individual classes or macro average. Is there some reason the ...
0
votes
0answers
32 views
Why is ROC a ranking metric and how do we understand it as a c-statistic?
I have been stuck on this topic for quite a long time. I still do not understand why AUROC is a ranking metric. In particular, what does "ranking" mean in AUROC? When I tried to code ROC out ...
0
votes
0answers
12 views
Can you do hyperparameter tuning using a PR curve? Moreover, would this still be considered a "PR curve"?
I am creating a graphical representation of PR results across various hyperparameter changes in an imbalanced dataset (model used was an SVM). I'm wondering if one would still consider this a "PR ...
0
votes
0answers
15 views
Why does using probabilities result in higher ROC_AUC scores?
in the most recent playground competition on kaggle (https://www.kaggle.com/c/tabular-playground-series-mar-2021/overview/evaluation) we once again have an evaluation via the area under the roc curve.
...
0
votes
1answer
20 views
imbalanced classes: ROC_AUC vs Precision_Recall AUC
I am dealing with a highly imbalanced classes problem. Accuracy is of course not a good performance metric in such cases, So I want to calculate either ROC AUC sore ...
2
votes
1answer
48 views
Why is the diagonal line in ROC space called the random guess?
all. I'm a student who studies machine learning.
When I found the wiki over the ROC(Reciever Operation Characteristic), I had one question.
According to the wiki, ROC space has a diagonal line and ...
1
vote
0answers
117 views
ROC AUC for multiclass problem
Just some quick questions to clarify my doubt please.
I know that one can get precision/recall for each class in a multiclass problem, e.g. in this classification ...
0
votes
0answers
124 views
Interpretation of Youden's J Statistic
Youden's J statistic is defined as J = sensitivity + specificity - 1, and is equal to the vertical distance between the chance line and the ROC curve for a classifier. I'm having trouble describing ...
