Receiver Operating Characteristic, also known as ROC curve.

learn more… | top users | synonyms

0
votes
0answers
20 views

The meaning of “training accuracy”?

If I split my data set into testing, training (further separated into subtraining and validation data set in cross-validation). In the context of machine learning and esp. in those ROC comparing the ...
0
votes
0answers
15 views

Why accuracy is equal to auROC in all of my data sets? [on hold]

Hi i have 10 datasets and applied given code to calculate accuracy and auROC but results are exactly same (upto 5 di) on all datasets. function EVAL = PreRecSenSpeFmeasurFun(ACTUAL,PREDICTED) % This ...
0
votes
0answers
29 views

Logistic Regression interpreting results in order of importance [closed]

For prediction model using Binary Logistic Regression, is there a best sequence to interpret the resulting logistic regression output to decide whether the results are good or not? In order to select ...
3
votes
1answer
77 views

ROC curve drawbacks

In the class yesterday, we were taught about logistic and subsequently the ROC curve and how to use it. My questions are: Is this the most common way to identify if the logistic model is the ...
1
vote
0answers
19 views

Equivalence of dominant classifier performance in ROC and PR spaces

In their article of 2006, The Relationship Between Precision-Recall and ROC Curves, J. Davis & M. Goadrich show that ROC (TPR vs FPR) and Precision-Recall (PR) curves have one-to-one ...
0
votes
0answers
6 views

Minimum sample size to establish a cutoff point

I conducted a case-control study (obese and normal body weight). I want to conduct ROC curve between my variables to establish a cutoff point for vitamin depending on insulin resistance risk. Does it ...
0
votes
1answer
18 views

Performance evaluation measures for binary random forest classifier

I am using Random forest (Matlab) to classify the binary data. Broadly, the input to the random forest is number of features and class label. And random forest, after training, return the labels for ...
0
votes
0answers
47 views

Leave-one-out cross validation output interpretation and ROC curve

I have taken plenty of time to try and help myself, but I keep reaching dead ends. I have a dataset consisting of body measurements collected from a bird species, and the sex of each bird (known by ...
0
votes
0answers
27 views

R - Plotting a ROC curve for a Naive Bayes classifier using ROCR. Not sure if I'm plotting it correctly

I have a Naive Bayes classifiers that I'm using to try to predict whether a game is going to win or lose based on historical data. The model has 25 variables in total, all of which are categorical ...
1
vote
0answers
50 views

A strange ROC curve

I produced the ROC curve for a continuous biomarker used as a classifier toward a binary class (absence/presence of a disease according to another biomarker), using the ROCR package in R. I have a ...
20
votes
4answers
3k views

What is the name of this chart showing false and true positive rates and how is it generated?

The image below shows a continuous curve of false positive rates vs. true positive rates: However, what I don't immediately get is how these rates are being calculated. If a method is applied to a ...
3
votes
2answers
104 views

Diagonal in ROC plot?

I've used the R ROCR package to generate an ROC plot. I can see a diagonal in my plot. However, the ROC curve that I see other people generated has only horizontal and vertical lines. Why would I get ...
0
votes
0answers
35 views

Use AUC for model comparison but what is the optimal threshold for final prediction

We can compare the performance of different models using AUC ROC and pick the one with large AUC. Then, we still need to choose and use specific threshold to predict the label for the test data. I ...
0
votes
0answers
18 views

In the classification framework, is AUROC a performance measure or metric?

I guess, the title is self-explaining. I have seen both so far and was wondering if there is a correct term or whether it does not really matter.
4
votes
1answer
38 views

Why two interpretations of AUC(area under the ROC curver) Equivalent

I found there are two ways to understand what AUC stands for but I couldn't get why these two interpretations are equivalent mathematically. In the first interpretation, AUC is the area under the ...
0
votes
0answers
14 views

Can I use p-value as the criteria to generate ROC cuvers

I developed an algorithm to predict positive and negative samples. The algorithm returns a p-value for each sample. Can I use the p-value (ranging from 0 to 1) as the criteria to generate a ROC curve? ...
0
votes
0answers
34 views

ROC curve smoothening

I conducted a logistic regression analysis in SPSS which resulted with a very good model. However when I wanted to do the ROC curve, it was drawn using only three points (see figure). All the examples ...
0
votes
1answer
33 views

Why is ROC curve always inreasing (or non-decreasing)?

As far as I understood, we plot the (False positive Rate, True Positive Rate) points for all values of threshold. Why should this curve be increasing always? (or non-decreasing)
17
votes
1answer
658 views

Did I just invent a Bayesian method for analysis of ROC curves?

Preamble This is a long post. If you're re-reading this, please note that I've revised the question portion, though the background material remains the same. Additionally, I believe that I've devised ...
0
votes
1answer
38 views

Is it legitimate to use sensitivity and specificity next to more proper performance measures to compare classifiers?

Clearly, Brier Score and AUROC are better performance measures to compare classifiers. However, besides that, I am interested in a let's call it more economic view. I could imagine a classifier being ...
0
votes
1answer
43 views

How to draw the ROC curve?

Currently I'm asking me how to draw the ROC curve (Receiver Operating Characteristic curve). On the x-axis there is the false positive rate (1-specificity) and on the y-axis there is the true positive ...
0
votes
0answers
29 views

Is there a package in R allowing to test whether a ROC curve is dominating the other?

Among other measures, I'd like to compare ROC curves for 2 classification methods. Is there a way to test automatically if one classifier performs better independent of the threshold? Is there a ...
0
votes
4answers
76 views

How is an ROC curve constructed for a set of data?

I know that sensitivity and recall (and precision, and ...
1
vote
1answer
81 views

Do these Precision-Recall (PR) curves indicate good classification performances?

I have trained a classifier for 3 different classes, the test datasets of which are imbalanced, and then plotted the PR curves (below) to evaluate their accuracies. The plots contain the number of ...
1
vote
1answer
44 views

Should PR AUC be used in cases where there is less than 5 positives vs 10000+ negatives?

I understand that the PR-AUC provides a better accuracy estimate than the ROC-AUC in the case of highly skewed datasets. But if I have a test dataset with less than 5 positives and 10000+ negatives, ...
3
votes
0answers
33 views

What is the acceptable event rate to use ROC-AUC instead of precision-recall curve?

It says here However, when dealing with highly skewed datasets, Precision-Recall (PR) curves give a more informative picture of an algorithm's performance. My question is; What is the common ...
2
votes
2answers
72 views

Average ROC for repeated 10-fold cross validation with probability estimates

I am planning to use repeated (10 times) stratified 10-fold cross validation on about 10,000 cases using machine learning algorithm. Each time the repetition will be done with different random ...
0
votes
0answers
17 views

Costs, Error rates, and Operating Point read from ROC

I am reading an article regarding ROC curves, Receiver Operating Characteristics (ROC). I feel I need a little bit of help here. My ultimate goal is to design an optimum classifier while there is ...
6
votes
2answers
96 views

How can I assess performance of a semi-supervised learning method?

I'm working with a semi-supervised learning task, where I only have positive and unlabelled data (PU learning). I've tested a few algorithms and would like to assess their performance. For ...
1
vote
1answer
50 views

Evaluation of diagnostic test ($R^2$ vs AUC)

A sort of a risk evaluation system is offered to us (sort of advertising). The output value is lethality (yes/no). The evaluators are several diagnoses, emergency status, age etc. No detailed ...
1
vote
0answers
29 views

Determining the validity of ROC curves

ROC curves were generated using ROCR package using only actual and predicted labels(from a SVM) as input. Script: ...
0
votes
0answers
18 views

WEKA / Binary classifiers: Why two AUCs?

If I use different classification algorithms in WEKA, one possible output is the ROC-AUC. Why do I get two AUC indicators, one for the positive instances and one for the negative instances (besides ...
0
votes
0answers
21 views

Logistic Regression Diagnostics

I am a bit confused how to assess the results of a logistic regression procedure I ran. Using SAS (on a 60% training set) - The LRT, Score and Wald test and got very low p-values. However, this model ...
0
votes
1answer
122 views

Scoring a classifier with ROC AUC

I'm confused about how scikit-learn's roc_auc_score is working. As I understand it, an ROC AUC score for a classifier is obtained as follows: Sample from the parameter space Fit the model Make ...
1
vote
1answer
71 views

How to insert my data in SPSS for Roc curve [closed]

I want to draw a roc curve in SPSS and I don't know how to insert my data in SPSS. The data I have are: ...
0
votes
1answer
14 views

Is there a name for the precision vs sensitivity plot?

For a particular task, instead of a standard ROC curve (which measures the sensitivity vs the specificity), I've found it useful to plot the precision vs the sensitivity (recall). Is there a name ...
2
votes
0answers
40 views

How to derive a mathematical formula for AUC?

Why the area under the ROC curve is the probability that a classifier will rank a randomly chosen "positive" instance (from the retrieved predictions) higher than a randomly chosen "positive" one ...
0
votes
0answers
11 views

How to draw a ROC curve for multi-class classification in Matlab? [duplicate]

I have predicted probability values for a dependent variable using logistic regression. By looking at the predicted probabilities for the training data set, I have divided the probabilities into three ...
0
votes
1answer
30 views

Reporting result of classifiers [duplicate]

I am using a binary classifier in my research. I wanna report the obtained result of classifier, but I am not sure about it. The learning prosedure is as follows: ...
1
vote
1answer
35 views

Relationship between pseudo-$R^2$ and area under the ROC curve

The $R^2$ of a model measures how well a model fits the data and is a measure of the shared variation between two (or more) variables. Its equivalent measure for logistic regression is the ...
0
votes
1answer
105 views

Area Under the ROC Curve, a simple question

I split my dataset into 2 parts: 75% of it is the training set, 25% of it is the test set. Then I estimated the logistic regression parameters in the training set and I compute the Area Under the ROC ...
0
votes
1answer
35 views

Finding Confusion Matrix from TPR and FPR

The confusion matrix is made up of TN, FN, FP, TP. Given ...
0
votes
1answer
49 views

ROC analysis and death/recurrence as binary marker

I am doing ROC-curve analysis on my patient cohort, and I am wondering if it is statistically ok to use "death" and "recurrence" as the binary marker, even though these parameters will be influenced ...
3
votes
1answer
67 views

Different ROC value for different packages in R, which one is correct?

I noticed that computing ROC with caret package and PROC packege sometimes gives different results. Usually they are the same, but if the predictions are worse than chance, caret will flip them and ...
0
votes
1answer
35 views

Manual construction of ROC rurve

I have read several articles about how to build a ROC curve but there is something I do not understand. Assuming I have a binary classifier, to draw a ROC curve I have to retrieve a list of scores ...
1
vote
1answer
41 views

ROC sensitivity and specificity

In all the ROC curves I've seen the sensitivity increase as the (1-specificity) increases, is this always the case? I guess my question can be summarized as: can the fpr be higher than the tpr?
0
votes
0answers
62 views

Is the ROC curve and estimated AUC from SPSS parametric or nonparametric?

I am using SPSS to generate some ROC curves, AUC and p values. According to SPSS manual, the AUC can be computed parametrically or nonparametrically. However, I do not see any option for that. There ...
0
votes
0answers
11 views

Combined ROC curve for 2 independent variables

I have two clinical tests and want to produce a single ROC curve based on a combined result from the two. How do I do this
0
votes
1answer
47 views

Plotting ROC in matlab

I plotted two Gaussian distribution of these groups of data with mean M_i and variance V_i corresponding to +1 and -1. ...
0
votes
0answers
7 views

ROC analysis for varying intervals between measure and event

I have a question about ROC analysis. The data looks like the following: A longitudinal study over 7 years. At baseline, measures of cardiovascular health were taken and combined into a metric where ...