Receiver Operating Characteristic, also known as ROC curve.

learn more… | top users | synonyms

0
votes
0answers
5 views

mlogitroc problem with Stata 13 [migrated]

I have a question about the additional module "mlogitroc", which should plot a ROC curve based on a multinomial logistic regression. More in details, at the end of computation the software displays ...
1
vote
1answer
16 views

ROC curve when the numbers of both total positives and negatives are unknown

I have to use a roc curve to find an agreement between sensitivity between the FPR and TPR. The problem is that I only know the true positives. How could I circumvent this? More accurate, I have a ...
0
votes
0answers
15 views

How to know if my data is balanced or imbalanced for an ROC curve analysis?

I am doing a research on the reliability of different models in detecting hidden defects in a test specimen. I have made a test specimen with defect prevalence about 25% (12 positive out of 49 total ...
0
votes
1answer
9 views

What does ROC-EER in percent stand for?

Ive tried to understand what the ROC Curve represents and what EER (Equal Error Rate) means. And I somehow think I got to understand some of the explanations I read on the internet and videos I ...
4
votes
2answers
89 views

d prime with 100% hit rate probability and 0% false alarm probability

I would like to calculate d prime for a memory task that involves detecting old and new items. The problem I have is that some of the subjects have hit rate of 1 and/or false alarm rate of 0, which ...
2
votes
1answer
45 views

One model performs better than the other. How to measure if it is statistically significant?

So, let's say that I train two models on the same dataset. I run the experiment once and I get the following results: Using a Neural Network I get an AUC ROC of 0.941. Using Random Forest I get an ...
4
votes
1answer
89 views

Is the objective to beat a random classifier when the data set is skewed using PR curves?

I have a testing data set where 1/3 of the observations are class-1 objects and the remainder class-0. Hence, the data set is skewed (skewed classifier), literature suggests that if the data set is ...
1
vote
1answer
46 views

Threshold selection by intersection of Sensitivity and Specificity

Some days ago, I learned in a lecture that the intersection of Sensitivity and Specificity provides an optimal compromise for choosing a classification threshold for logit or probit models. However, ...
0
votes
0answers
15 views

Recommended performance metrics of a binary classifier having an example-based cost matrix

I would like to know what are the recommended performance metrics to assess a binary classifier, when the cost matrix is changing for each sample. My problem comes from the fact that in this case, ...
2
votes
0answers
26 views

does the area under the curve (AUC) has some interpretation? [duplicate]

I know that the ROC curve plots true positives vs false positives, but does it have any other interpretation, or is it just an arbitrary performance measure? Also, in the case of very unbalanced ...
17
votes
4answers
896 views

What does AUC stand for and what is it?

Searched high and low and have not been able to find out what AUC, as in related to prediction, stands for or means.
1
vote
1answer
37 views

ROC curves for unbalanced datasets

Consider an input matrix $X$ and a binary output $y$. A common way to measure the performance of a classifier is to use ROC curves. In a ROC plot the diagonal is the result that would be obtained ...
0
votes
0answers
14 views

F1 Score vs ROC [duplicate]

How would one decide which performance measure to use when comparing classification algorithms for prediction purposes? Either the F-Score (Precision,Recall) or ROC/AUC analysis or both.
1
vote
1answer
80 views

how to decide which logistic regression model is better?

I have the following 3 models: ...
1
vote
0answers
18 views

When to use the Kappa statistic evaluation metric?

Can someone tell me when is it appropriate to use the Kappa statistic? Also why to use it when one can use Area Under the ROC curve? Or even the Area under the precision-recall curve? So what are the ...
0
votes
0answers
22 views

Accuracy of rpart for categorical

Below is an example of fitting categorical data using rpart. But how to compare the predictions from rpart with the actual data? Also, is it possible to draw a ROC curve for the testing and training ...
0
votes
0answers
63 views

How to evaluate the optimal cutoff of ROC curve related to logistic regression using roc from the R package pROC?

I would like to get the optimal cutoff of an ROC curve relating to a logistic regression. I am using the roc from the R package pROC. I am assuming same cost of false negative and false positive using ...
0
votes
1answer
18 views

Minimum number of tested patients to have a reasonable ROC curve [closed]

What are the minimum number of tested patients and the acceptable prevalence percent required to have a reasonable ROC curve? for example, can I test a total of 16 patients, 5 are diseased and 11 are ...
4
votes
1answer
39 views

Is up- or down-sampling imbalanced data actually that effective? Why?

I frequently hear up- or down-sampling of data discussed as a way of dealing with classification of imbalanced data. I understand that this could be useful if you're working with a binary (as opposed ...
0
votes
0answers
24 views

Decision function for BernoulliNB classifier. ( for use in plotting ROC and PR curves )

I would like to plot the PR curve using scikit-learn for the Bernoulli Naive Bayes estimator. However, attempting to do so give an error, ...
0
votes
1answer
22 views

Affect of Misclassification Cost on SVM

I am using Matlab to train an SVM for very unbalanced data. However, my concern is not so much for the particular class assignment (ie 1/0), but rather to the scores (the prethreshold continuous SVM ...
1
vote
3answers
238 views

How to interpret a ROC curve?

I applied logistic regression to my data on SAS and here are the ROC curve and classification table. I am comfortable with the figures in the classification table, but not exactly sure what the roc ...
2
votes
3answers
109 views

Is there any other measure of the performance of a classifier than the area under the ROC curve?

I am trying to draw an ROC curve for a classifier and wondered to know if there is any other measure for the performance of the classifier than the AUC. And is there any free software that I can use ...
3
votes
0answers
53 views

Connections between d' (d-prime) and AUC (Area Under the ROC Curve); underlying assumptions

In machine learning we may use the area under the ROC curve (often abbreviated AUC, or AUROC) to summarise how well a system can discriminate between two categories. In signal detection theory often ...
1
vote
1answer
54 views

Testing Logistic Regression Classifier in R

I am testing the logistic regression classifier in R. I created some test data like this: x=runif(10000) y=runif(10000) df=data.frame(x,y,as.factor(x-y>0)) ...
0
votes
1answer
32 views

ROC / AUC for polynominal Labels

How can I calculate the Area Under Curve for a classifier of a plynominal label in Rapidminer? I could only find a performance operator for binominal labels that provides the AUC value.
0
votes
0answers
70 views

How do we generate the ROC curve for Linear Discriminant Analysis method

I know the method to generate the ROC curve for other methods such as naive Bayes where the tuning parameter is the threshold like also in logistic regression. If we want to generate the ROC curve ...
1
vote
1answer
83 views

Better in ROC AUC vs. better in PR AUC

I'm comparing two classification models by computing the area under ROC and Precision-Recall curves. However sometimes one model is better with AU-ROC but worse in AU-PR, and other times it's better ...
0
votes
1answer
55 views

What is AUC of PR-curve?

I understand that AUC under ROC curve is a classic evaluation measurement for classifiers (which is basically the accuracy). However, when data is imbalanced, PR will be alternative. So, what does the ...
1
vote
2answers
82 views

Help with understanding statistical measures and Receiver Operating Characteristics

In my machine learning class we just went over statistical measures and plots. We looked at the definitions of True Positive Rate (sensitivity/recall, etc), 1-False Positive Rate (speciicity), ...
1
vote
1answer
55 views

Why AUC-PR increases when the number of positives increase?

I asked a question earlier about comparing models using Precision-Recall AUC. One of the answers included the following statement: "The larger the fraction of positives in the data set, the larger the ...
1
vote
1answer
59 views

Ranking two models based on ROC-AUC and PR-AUC

I have two methods/classifiers (completely different models) that I need to decide which one is better. The dataset is imbalanced. I trained both classifiers on the same dataset and then I computed ...
2
votes
1answer
60 views

Is it correct to use Precision-Recall AUC in a balanced dataset situation?

I have a binary classification scenario with a dataset that is unbalanced (much more negatives than positives). When I train a classifier on this dataset I get a Precision-Recall AUC of 0.7. Then I ...
0
votes
0answers
79 views

How to calculate the area under the precision-recall curve for the random classifier?

I know that the random classifier score in ROC AUC (Area under the curve) is always 0.5. My question is: how to calculate the Area under the precision-recall curve for the random classifier?
0
votes
2answers
107 views

What is the convex hull in ROC curve?

I'm reading a paper about ROC and PR curves. They mentioned the ROC convex hull but they don't define it or say what it is. Can someone please tell me the meaning of it? What is a convex hull in ROC ...
2
votes
2answers
74 views

An intuitive meaning of the area under the PR curve?

Wikipedia says that an interpretation of the area under the ROC curve is: "the area under the curve is equal to the probability that a classifier will rank a randomly chosen positive instance higher ...
1
vote
1answer
260 views

Finding true positive / negative and false positive / negative rates using R

I have a data frame with two classes. I want to find the true positive and false positive rate and then plot the ROC curve. I tried this: ...
0
votes
0answers
11 views

Tree classifier or nested model?

I am new to statistics and was wondering what the right kind of model to use for the following scenario. I have two sets of continuous observations A and B for 50 samples. These 50 samples are ...
0
votes
2answers
51 views

How to turn my data into a ROC curve in R? [closed]

I'd like to generate ROC curves in R, but I'm confused about what input to give either ROCR or pROC. I have 5000 data-points for which I know the true classification (1 or 0), and a continuous ...
0
votes
0answers
70 views

Complete Logistic Regression framework using K-Cross validation

I'm implementing a logistic regression model in a low event rate data. I have gone through many webpages (including stackoverflow, including my questions) but none answer or describe the end-to-end ...
0
votes
0answers
54 views

Significance of a dichotomized variable from a continuous variable

I am analyzing a X continuous independent variable with a Y binary response. The investigator has interested on dichotomize the X variable by the “best” cutpoint from the ROC curve (for example the ...
0
votes
2answers
101 views

ROC (Epi library) how to calculate TP, FN,TN, FP

I'm trying to find how to compute the true negative (TN), false negative (FN), true positive (TP), and false positive (FP) if I have a cutpoint like in the following picture: ...
0
votes
2answers
42 views

Use of data from ROC curve

In order to find an optimal time for initiation of treatment post surgery (oncologic patients) I created a ROC curve with death defined as event. The AUC was not significant. However, I decided to use ...
0
votes
0answers
30 views

Is it logical to stand on the chance-line (50-50 %) when we don't know a-priori probabilities?

Let we have two hypotheses $H_0$ and $H_1$ and we don't know their a-priori probabilities. If we wish to calculate the average probability of error, does it makes sense to assume 50-50 % chance of ...
0
votes
0answers
47 views

How to evaluate a trained model using parameters other than AUC in RapidMiner?

I am using RapidMiner to build predictive models trained and cross-validated by a set of medical data(65 cases. 18 attributes), I am now running trials by trying different combinations of learners and ...
0
votes
0answers
56 views

Cutoff and precision values of a binary classifier

Let's say I have fitted a binary classifier to some data and I'm varying the cutoff value, effectively producing a ROC-curve. Knowing the true proportions of positives and negatives, I can calculate ...
2
votes
2answers
113 views

ROC for more than 2 outcome categories

How do you construct ROC Curves when there are more than two outcome categories (in my case, I have four)? I've heard you should do this for the most popular group. Are there any other ideas? Are ...
0
votes
0answers
38 views

Selecting individuals from a population using a binary classifier

I have a dataset consisting of around 200 individuals, whose outcome is either of state $0$ or $1$. I am able to make binary classifiers and predictors on this set and build ROC-curves for them just ...
1
vote
3answers
141 views

ROC graph shape

Could you explain to me how the shape of a ROC curve is determined? From the following illustration, it seems that for every time the actual class (C) is positive, it goes up and when it's negative, ...
2
votes
0answers
157 views

Unbalanced dataset - ROC curve to compare classifiers?

I use the machine learning software WEKA for data mining on biological data. I would describe my dataset as unbalanced: It comprises around 2000 instances, ...