Receiver Operating Characteristic, also known as ROC curve.

learn more… | top users | synonyms

1
vote
0answers
24 views

R coding for ROC [migrated]

I am having a hard time translating the example command for an ROC to my dataset. This is for the pROC package This is the example using data (aSAH) ...
1
vote
2answers
30 views

Using ROC curve for balanced data

I understand that using the area under the ROC curve is a useful error measurement for unbalanced data. What happens if we use it for balanced data?
1
vote
0answers
10 views

How can I calculate AUC using Gini coefficient?

In the Gini Coefficient's Wikipedia page, it is defined as $G= 1 - \frac{\Sigma_{i=1}^n f(y_i)(S_{i-1}+S_i)}{S_n}$ for discrete variables, where $S_i= \Sigma_{j=1}^i f(y_i)y_i$ and $S_0=0$ ($y$ being ...
0
votes
0answers
6 views

Error in plot.new() : figure margins too large [migrated]

I'm currently developing a ROC plot and got the following error: Error in plot.new() : figure margins too large Here's the code that I'm using: library(ROCR) ...
1
vote
0answers
24 views

Calculating two-tailed p-value from z-score for ROC AUC comparison

I am comparing two predictive models by their bootstrapped ROC AUCs with the method originally described by Hanley and McNeil and modified for bootstrapped data by Robin et al. I'm calculating the ...
0
votes
0answers
5 views

mlogitroc problem with Stata 13 [migrated]

I have a question about the additional module "mlogitroc", which should plot a ROC curve based on a multinomial logistic regression. More in details, at the end of computation the software displays ...
1
vote
1answer
24 views

ROC curve when the numbers of both total positives and negatives are unknown

I have to use a roc curve to find an agreement between sensitivity between the FPR and TPR. The problem is that I only know the true positives. How could I circumvent this? More accurate, I have a ...
0
votes
1answer
34 views

How to know if my data is balanced or imbalanced for an ROC curve analysis?

I am doing a research on the reliability of different models in detecting hidden defects in a test specimen. I have made a test specimen with defect prevalence about 25% (12 positive out of 49 total ...
0
votes
1answer
11 views

What does ROC-EER in percent stand for?

Ive tried to understand what the ROC Curve represents and what EER (Equal Error Rate) means. And I somehow think I got to understand some of the explanations I read on the internet and videos I ...
4
votes
2answers
167 views

d prime with 100% hit rate probability and 0% false alarm probability

I would like to calculate d prime for a memory task that involves detecting old and new items. The problem I have is that some of the subjects have hit rate of 1 and/or false alarm rate of 0, which ...
2
votes
1answer
58 views

One model performs better than the other. How to measure if it is statistically significant?

So, let's say that I train two models on the same dataset. I run the experiment once and I get the following results: Using a Neural Network I get an AUC ROC of 0.941. Using Random Forest I get an ...
4
votes
1answer
101 views

Is the objective to beat a random classifier when the data set is skewed using PR curves?

I have a testing data set where 1/3 of the observations are class-1 objects and the remainder class-0. Hence, the data set is skewed (skewed classifier), literature suggests that if the data set is ...
1
vote
1answer
69 views

Threshold selection by intersection of Sensitivity and Specificity

Some days ago, I learned in a lecture that the intersection of Sensitivity and Specificity provides an optimal compromise for choosing a classification threshold for logit or probit models. However, ...
0
votes
0answers
19 views

Recommended performance metrics of a binary classifier having an example-based cost matrix

I would like to know what are the recommended performance metrics to assess a binary classifier, when the cost matrix is changing for each sample. My problem comes from the fact that in this case, ...
2
votes
0answers
28 views

does the area under the curve (AUC) has some interpretation? [duplicate]

I know that the ROC curve plots true positives vs false positives, but does it have any other interpretation, or is it just an arbitrary performance measure? Also, in the case of very unbalanced ...
17
votes
4answers
986 views

What does AUC stand for and what is it?

Searched high and low and have not been able to find out what AUC, as in related to prediction, stands for or means.
1
vote
1answer
53 views

ROC curves for unbalanced datasets

Consider an input matrix $X$ and a binary output $y$. A common way to measure the performance of a classifier is to use ROC curves. In a ROC plot the diagonal is the result that would be obtained ...
0
votes
0answers
15 views

F1 Score vs ROC [duplicate]

How would one decide which performance measure to use when comparing classification algorithms for prediction purposes? Either the F-Score (Precision,Recall) or ROC/AUC analysis or both.
1
vote
1answer
94 views

how to decide which logistic regression model is better?

I have the following 3 models: ...
1
vote
0answers
27 views

When to use the Kappa statistic evaluation metric?

Can someone tell me when is it appropriate to use the Kappa statistic? Also why to use it when one can use Area Under the ROC curve? Or even the Area under the precision-recall curve? So what are the ...
0
votes
0answers
38 views

Accuracy of rpart for categorical

Below is an example of fitting categorical data using rpart. But how to compare the predictions from rpart with the actual data? Also, is it possible to draw a ROC curve for the testing and training ...
0
votes
0answers
85 views

How to evaluate the optimal cutoff of ROC curve related to logistic regression using roc from the R package pROC?

I would like to get the optimal cutoff of an ROC curve relating to a logistic regression. I am using the roc from the R package pROC. I am assuming same cost of false negative and false positive using ...
0
votes
1answer
24 views

Minimum number of tested patients to have a reasonable ROC curve [closed]

What are the minimum number of tested patients and the acceptable prevalence percent required to have a reasonable ROC curve? for example, can I test a total of 16 patients, 5 are diseased and 11 are ...
4
votes
1answer
49 views

Is up- or down-sampling imbalanced data actually that effective? Why?

I frequently hear up- or down-sampling of data discussed as a way of dealing with classification of imbalanced data. I understand that this could be useful if you're working with a binary (as opposed ...
0
votes
0answers
33 views

Decision function for BernoulliNB classifier. ( for use in plotting ROC and PR curves )

I would like to plot the PR curve using scikit-learn for the Bernoulli Naive Bayes estimator. However, attempting to do so give an error, ...
0
votes
1answer
22 views

Affect of Misclassification Cost on SVM

I am using Matlab to train an SVM for very unbalanced data. However, my concern is not so much for the particular class assignment (ie 1/0), but rather to the scores (the prethreshold continuous SVM ...
1
vote
3answers
270 views

How to interpret a ROC curve?

I applied logistic regression to my data on SAS and here are the ROC curve and classification table. I am comfortable with the figures in the classification table, but not exactly sure what the roc ...
2
votes
3answers
121 views

Is there any other measure of the performance of a classifier than the area under the ROC curve?

I am trying to draw an ROC curve for a classifier and wondered to know if there is any other measure for the performance of the classifier than the AUC. And is there any free software that I can use ...
3
votes
0answers
70 views

Connections between d' (d-prime) and AUC (Area Under the ROC Curve); underlying assumptions

In machine learning we may use the area under the ROC curve (often abbreviated AUC, or AUROC) to summarise how well a system can discriminate between two categories. In signal detection theory often ...
1
vote
1answer
63 views

Testing Logistic Regression Classifier in R

I am testing the logistic regression classifier in R. I created some test data like this: x=runif(10000) y=runif(10000) df=data.frame(x,y,as.factor(x-y>0)) ...
0
votes
1answer
45 views

ROC / AUC for polynominal Labels

How can I calculate the Area Under Curve for a classifier of a plynominal label in Rapidminer? I could only find a performance operator for binominal labels that provides the AUC value.
0
votes
0answers
101 views

How do we generate the ROC curve for Linear Discriminant Analysis method

I know the method to generate the ROC curve for other methods such as naive Bayes where the tuning parameter is the threshold like also in logistic regression. If we want to generate the ROC curve ...
2
votes
2answers
110 views

Better in ROC AUC vs. better in PR AUC

I'm comparing two classification models by computing the area under ROC and Precision-Recall curves. However sometimes one model is better with AU-ROC but worse in AU-PR, and other times it's better ...
0
votes
1answer
63 views

What is AUC of PR-curve?

I understand that AUC under ROC curve is a classic evaluation measurement for classifiers (which is basically the accuracy). However, when data is imbalanced, PR will be alternative. So, what does the ...
1
vote
2answers
91 views

Help with understanding statistical measures and Receiver Operating Characteristics

In my machine learning class we just went over statistical measures and plots. We looked at the definitions of True Positive Rate (sensitivity/recall, etc), 1-False Positive Rate (speciicity), ...
1
vote
1answer
68 views

Why AUC-PR increases when the number of positives increase?

I asked a question earlier about comparing models using Precision-Recall AUC. One of the answers included the following statement: "The larger the fraction of positives in the data set, the larger the ...
1
vote
1answer
69 views

Ranking two models based on ROC-AUC and PR-AUC

I have two methods/classifiers (completely different models) that I need to decide which one is better. The dataset is imbalanced. I trained both classifiers on the same dataset and then I computed ...
2
votes
1answer
75 views

Is it correct to use Precision-Recall AUC in a balanced dataset situation?

I have a binary classification scenario with a dataset that is unbalanced (much more negatives than positives). When I train a classifier on this dataset I get a Precision-Recall AUC of 0.7. Then I ...
0
votes
0answers
100 views

How to calculate the area under the precision-recall curve for the random classifier?

I know that the random classifier score in ROC AUC (Area under the curve) is always 0.5. My question is: how to calculate the Area under the precision-recall curve for the random classifier?
0
votes
2answers
140 views

What is the convex hull in ROC curve?

I'm reading a paper about ROC and PR curves. They mentioned the ROC convex hull but they don't define it or say what it is. Can someone please tell me the meaning of it? What is a convex hull in ROC ...
2
votes
2answers
86 views

An intuitive meaning of the area under the PR curve?

Wikipedia says that an interpretation of the area under the ROC curve is: "the area under the curve is equal to the probability that a classifier will rank a randomly chosen positive instance higher ...
1
vote
1answer
363 views

Finding true positive / negative and false positive / negative rates using R

I have a data frame with two classes. I want to find the true positive and false positive rate and then plot the ROC curve. I tried this: ...
0
votes
0answers
11 views

Tree classifier or nested model?

I am new to statistics and was wondering what the right kind of model to use for the following scenario. I have two sets of continuous observations A and B for 50 samples. These 50 samples are ...
0
votes
2answers
56 views

How to turn my data into a ROC curve in R? [closed]

I'd like to generate ROC curves in R, but I'm confused about what input to give either ROCR or pROC. I have 5000 data-points for which I know the true classification (1 or 0), and a continuous ...
0
votes
0answers
71 views

Complete Logistic Regression framework using K-Cross validation

I'm implementing a logistic regression model in a low event rate data. I have gone through many webpages (including stackoverflow, including my questions) but none answer or describe the end-to-end ...
0
votes
0answers
66 views

Significance of a dichotomized variable from a continuous variable

I am analyzing a X continuous independent variable with a Y binary response. The investigator has interested on dichotomize the X variable by the “best” cutpoint from the ROC curve (for example the ...
0
votes
2answers
123 views

ROC (Epi library) how to calculate TP, FN,TN, FP

I'm trying to find how to compute the true negative (TN), false negative (FN), true positive (TP), and false positive (FP) if I have a cutpoint like in the following picture: ...
0
votes
2answers
43 views

Use of data from ROC curve

In order to find an optimal time for initiation of treatment post surgery (oncologic patients) I created a ROC curve with death defined as event. The AUC was not significant. However, I decided to use ...
0
votes
0answers
30 views

Is it logical to stand on the chance-line (50-50 %) when we don't know a-priori probabilities?

Let we have two hypotheses $H_0$ and $H_1$ and we don't know their a-priori probabilities. If we wish to calculate the average probability of error, does it makes sense to assume 50-50 % chance of ...
0
votes
0answers
53 views

How to evaluate a trained model using parameters other than AUC in RapidMiner?

I am using RapidMiner to build predictive models trained and cross-validated by a set of medical data(65 cases. 18 attributes), I am now running trials by trying different combinations of learners and ...