# Questions tagged [roc]

Receiver Operating Characteristic, also known as the ROC curve.

721 questions
Filter by
Sorted by
Tagged with
13 views

### Panel Data in R - Can I implement a plm regression with categorical data?

I am working (In R) on a panel modelling of an econometric problem. I have : one dependent variable : Y with values in real positive numbers two explanatory variables : VAR_1 (real positive), VAR_2 (...
33 views

### ROC for testing goodness of fit

I'm interested in using ROC to test for goodness of fit for binary models such as logistic regression. I'm a bit confused by the literature where it is mostly just explained as a valid technique to ...
7 views

### Is it possible to determine the Probability of Error for the classifier form the ROC curve at a specific point?

Given a classifier with an ROC curve at a specific point, say: (FNR = 0.01, FPR = 0.1). Is it possible to determine the Probability of Error for the classifier at this point?
18 views

### Is it possible for a binary classifier to have lower accuracy, macrof1 and binaryf1 but higher ROC AUC? [duplicate]

I've got the results of two classifiers based on 5 different splits of training and testing sets. Their mean and std of the results are as follow: Method-------Accuracy -- MacroF1 -- BinaryF1---- ROC ...
22 views

### How to generate a ROC curve from ground truth and predictions (in R package pROC)?

I've trained several machine learning models (GLM, SVM, random forest) in R to perform binary classification (predicting the presence of gallstones) and plotted ROC curves/computed AUC using the pROC ...
27 views

### How can I better choose my model?

I've made several experimentations in a classification problem, I've got these results: ROC-AUC Metric: Train: 0.99, CV: 0.95 +/- 0.01, Test: 0.96 Train: 0.97, CV: 0.94 +/- 0.01, Test: 0.94 Train: 0....
17 views

### Treating “probability thresholds” (classification problems) as a hyperparameter

I found this link over here : https://topepo.github.io/caret/using-your-own-model-in-train.html (section 13.8) My understanding is that the probability threshold over here is essentially being ...
38 views

### Regression model comparison/validation in an independent cohort: other method than c-index

What I have: I developed a logistic regression model (M1) to predict lymph node metastasis in cancer patients using the variables A + B + C + D + E and a training/validation data set D1, E is the new ...
28 views

### ROC Curve for unbounded scores

Say I have a classifier that assigns a score to an image based on whether it has a cat in it. The higher the score, the more likely there's a cat in it. But for this classifier, the value of the score ...
21 views

### ROC and AUC for clustering algorithms [closed]

I am working on some clustering algorithms like DBSCAN and local outlier factor. Now i want to know how can I make ROC and AUC curves from clustering results. Do anyone know how can i make RO and AUC ...
14 views

### Identifying threshold from Youden Index - Using ROC curve to calculate minimally important change (MIC)

I am trying to determine the minimally important change (MIC) of a frailty instrument using an anchor-based approach outlined below. Step 1. Fit a logistic regression model between ...
12 views

### How to select best models if the ROC AUC score changes drastically at each separate run?

Below are two plots for ROC curves with their AUC mentioned in the legend brackets. How do I shortlist the best models if the scores differ at each run? Should I rather calculate the ROC AUC only from ...
32 views

### Random Forest with train AUC = 1 and test AUC = 58%

I'm trying to understand why my train AUC = 1 while my test AUC is near 58% using random forest. Context: You are trying to sell a product, and you have historic data about the purchases/noPurchases ...
31 views

### What is the name of the probability distribution of ROC-AUC when training machine learning models?

When training ML models like neural networks they are random initialized. That has the effect that the results (ROC-AUC for example) are influenced by random effect. When I train them multiple times ...
138 views

### How should two cross-validated logistic regression models be compared?

I'm using 100 times 10-fold repeated cross-validation to assess the ROC-AUC performance improvement of adding a biomarker to an existing model: Model_A : pred1 + pred2 Model_B :pred1 + pred2 + pred3 I'...
18 views

### What do the steps in ROC curve represent? [duplicate]

I have got the following ROC for a binary classification task using scikit-learn's Gaussian Naive Bayes classifier. The graph shown here shows the ROC curve and has various step like shape. I want to ...
15 views

### IntcensROC (R package) - same AUC regardless of input parameters

I have a data frame including various continuous variables (size of a lymph node measured in different ways) and an outcome (disease-free interval). I have used the intcensROC package to assess the ...
29 views

### How can an ROC AUC and its 95%CI be estimated from repeated crossvalidation?

I'm using repeated 100 times 10-fold cross-validation to provide internal validation of a logistic regression model. What is the best way to obtain the ROC AUC and it's 95% CI from this data. If I ...
28 views

### ROC Curves for Regression Output

I am working on a broad machine learning-based problem, which can be approached in several different ways. Essentially, my training values are floats between 0.0 and 1.0, and I have approached this in ...
9 views

### Help identifying Cumulative Expectation vs Cumulative Identification Graph

In a classification problem, I rank scored observations descending by (calibrated) probability of positive result. If I create a graph that shows cumulative % of total expectation captured on the Y ...
32 views

### Is there a way to get the optimal cutoff points based on probability of topic models and the outcomes?

I have topic models probability obtained using LDA topic models method. I’d like to use these probabilities for 5 topics to predict an ...
1k views

### Determine how good an AUC is (Area under the Curve of ROC)

I'm currently working on a project involving using different sets of data as a predictor to predict the outcome of out-sample data. I use AUC (Area under the Curve of ROC) to compare the performances ...
55 views

### AUC ROC and Varying Thresholds?

I understand that the ROC curve will plot the sensitivity vs FPR for varying thresholds. For my SVM ML model, I desire a good sensitivity score so I have decreased the threshold to make a positive ...
73 views

### Building AUC/ROC curve without probabilities, only with actual/predicted labels

If we dont have access to model and have just actual and predicted labels without probabilities, is it still be possible to plot AUC/ROC curve. For example can we have the curve from the following ...
35 views

### Can I draw the ROC curve from ROC_AUC Score?

The question may look rediculous. The problem in my hand is a multi-class (5 class labels) classification problem and I coded it a year back. I am having the resulsts such as G-mean, F-score, and ...
20 views

### How to create an ROC model using three classes

I am trying to create a ROC plot based on response variable with three classes. I believe that is possible based on the answer to this question Here. In the answer provide in this question though, ...
18 views

### ROC Curve for data sets with large negative bias

For context, I've read this forum here regarding a similar issue, and it seems the conclusion on there was that precision-recall curves are better-suited for data sets where there is a large negative ...
6 views

### Generating a continuous OD graph from a finite set of data points

This is an extract from Donald Bamber's paper regarding OD graphs and ROC plots. Bamber D. The area above the ordinal dominance graph and the area below the receiver operating characteristicgraph. ...
26 views

### Precision-recall curve for highly imbalanced multi-class classification task

I would like to measure the performance of my network on a multi-class classification problem and wanted to use the precision-recall curve. I have four classes, of which three are extremely ...
21 views

### One-sided test for AUC (area under the curve for ROC)

I am trying to determine the power of testing the AUC of a diagnostic test with continuous score against a one-sided alternative. I was thinking I could just use the confidence intervals produced by <...
46 views

### Selecting Threshhold from ROC curve [closed]

I plotted an ROC curve for a classification problem and I am looking a way to find out the threshold point for the left most top point - Highest TPR and low FPR. How do I do it in Python ?
21 views

### Can you cross validate a regression model using supervised learning?

Our team created a regression model based on a select data size/parameter and achieved a good C-statistic (area under ROC curve). We faced a few difficulties with submission for publication, so I ...
11 views

### Help interpreting my network output

I am having some issues getting my network trained well, and I would be grateful for some feedback. I have a neural network to perform binary classification. The final layer has one node and a sigmoid ...
13 views

### Query on Precision-Recall curve

I have an imbalanced dataset - Kaggle's Porto Seguro insurance dataset. I have applied Random Forest and XGBoost classifiers on the imbalanced dataset, under-sampled dataset and over-sampled dataset. ...
49 views

### Is the AUC an incoherent measure of classifier performance?

I'm learning about performance measures for binary classifiers. Reading about the AUC-ROC score I came across the article Measuring classifier performance: a coherent alternative to the area under the ...
31 views

### (multiple) Infinite/right censored independent values in diagnostics

We are facing a problem with infinite/right censored independent values. Case-control study where decay time (mono-exponential function) and half-value time seems to be of great diagnostic accuracy. ...
192 views

### How do I plot ROC curves with binary predictions?

Here is a simple dataset of actual and predicted results, with the resulting error matrix. How do I plot an ROC curve with this? I don't understand why the curve is not just four ordered pairs. ...
119 views

### Is my AUC too good to be true?

I've been going in circles for months...I want to generate a list of thresholds from my training data so I can see all the thresholds at every sensitivity/1-specificity of a model. You can do this ...
25 views

### threshold coded ROC curve interpretaion

I'm trying to figure out ROC curve better and must be kind of dumb but i don't know what are the thresholds of right side of image saying! On this graph, the y-axis is true positive rate, and the x-...
9 views

### How to remove the effect of age on the ROC curve? [duplicate]

I have data set, and the outcome variable is the prognosis (survival or death). I want to evaluate the prognosis of the patient by the SOFA scores, but the problem now is that the age of this data set ...
94 views

### Threshold ROC vs. Cut-Off in confusion matrix (binary regression)

I am trying to understand the link between the threshold in ROC-analysis and the threshold defined in classification table. Criterion is binary with 0 or 1. 1) Someone can determine a confusion ...
18 views

### Explaining a Conformal Prediction (vs NonConformal)

I'm trying to understand the definition of a conformal prediction and possibly the difference with a non-conformal prediction. What I get as a general idea is that the conformal prediction will be ...
42 views

### Cumulative Accuracy Profile(CAP) Curve vs Receiver Operator Characteristic (ROC) Curve

While going through several metrics such as Accuracy and Recall to measure the performance of a Machine Learning model, I found in some articles that ROC Curve and CAP Curve are great for ...
10 views

### ROC for a family of repeated experiments

Suppose I have a continuous covariate like age, and I want to use it for RNA expression which means that I have to fit a large number of linear models, one per gene, that predict the expression. ...
37 views

### How do I display my output comparing the effect of variables on classification of disease from a random forest analysis in R using an AUROC?

I am using a random forest classification to compare how the classification of disease improves when combining metabolites with two other measures ( visceral fat and CRP-1 levels) to see if adding the ...
51 views

### Plot ROC curve in R with the PRROC package

After many hours of research, trial and error and frustration I hope someone here will be able to guide me in the right direction. I am fairly new to R and statistics and can not wrap my hand about ...
24 views

### Calculating three average precisions and a single value for ROC from raw predicted class outputs

I'm not a statistician or mathematician so I apologize if I use any terms incorrectly. Please do point out any errors in my use of terminology. The four values I need are the equivalent of Weka's ROC ...
58 views

### Why can't my ROC reach a low PPV?

I am creating a couple of models (RF, SVM, LR) and I want to evaluate all of them on a certain PPV (0.7). This question and this question helped me write my code: ...