Receiver Operating Characteristic, also known as ROC curve.

learn more… | top users | synonyms

0
votes
0answers
7 views

How to plot ROC curve after using SMO classifier in weka? [on hold]

I classified my data using SMO function in weka. The ROC plot that I got is not a curve (it is based on just one threshold). can anyone help me to plot ROC curve for more decision threshold? how can ...
0
votes
0answers
7 views

Accuracy of rpart for categorical

Below is an example of fitting categorical data using rpart. But how to compare the predictions from rpart with the actual data? Also, is it possible to draw a ROC curve for the testing and training ...
0
votes
0answers
42 views

How to evaluate the optimal cutoff of ROC curve related to logistic regression using roc from the R package pROC?

I would like to get the optimal cutoff of an ROC curve relating to a logistic regression. I am using the roc from the R package pROC. I am assuming same cost of false negative and false positive using ...
0
votes
1answer
16 views

Minimum number of tested patients to have a reasonable ROC curve [closed]

What are the minimum number of tested patients and the acceptable prevalence percent required to have a reasonable ROC curve? for example, can I test a total of 16 patients, 5 are diseased and 11 are ...
4
votes
1answer
27 views

Is up- or down-sampling imbalanced data actually that effective? Why?

I frequently hear up- or down-sampling of data discussed as a way of dealing with classification of imbalanced data. I understand that this could be useful if you're working with a binary (as opposed ...
0
votes
0answers
15 views

Decision function for BernoulliNB classifier. ( for use in plotting ROC and PR curves )

I would like to plot the PR curve using scikit-learn for the Bernoulli Naive Bayes estimator. However, attempting to do so give an error, ...
0
votes
1answer
14 views

Affect of Misclassification Cost on SVM

I am using Matlab to train an SVM for very unbalanced data. However, my concern is not so much for the particular class assignment (ie 1/0), but rather to the scores (the prethreshold continuous SVM ...
0
votes
0answers
14 views

ROC curve for Nondestructive testing

I am working on Nondestructive testing of concrete bridge deck and would like to use the ROC curve to assess and compare the reliability of these NDT's. I made a grid on the deck and will test each ...
1
vote
3answers
205 views

How to interpret a ROC curve?

I applied logistic regression to my data on SAS and here are the ROC curve and classification table. I am comfortable with the figures in the classification table, but not exactly sure what the roc ...
2
votes
3answers
95 views

Is there any other measure of the performance of a classifier than the area under the ROC curve?

I am trying to draw an ROC curve for a classifier and wondered to know if there is any other measure for the performance of the classifier than the AUC. And is there any free software that I can use ...
3
votes
0answers
42 views

Connections between d' (d-prime) and AUC (Area Under the ROC Curve); underlying assumptions

In machine learning we may use the area under the ROC curve (often abbreviated AUC, or AUROC) to summarise how well a system can discriminate between two categories. In signal detection theory often ...
1
vote
1answer
44 views

Testing Logistic Regression Classifier in R

I am testing the logistic regression classifier in R. I created some test data like this: x=runif(10000) y=runif(10000) df=data.frame(x,y,as.factor(x-y>0)) ...
0
votes
1answer
24 views

ROC / AUC for polynominal Labels

How can I calculate the Area Under Curve for a classifier of a plynominal label in Rapidminer? I could only find a performance operator for binominal labels that provides the AUC value.
0
votes
0answers
45 views

How do we generate the ROC curve for Linear Discriminant Analysis method

I know the method to generate the ROC curve for other methods such as naive Bayes where the tuning parameter is the threshold like also in logistic regression. If we want to generate the ROC curve ...
1
vote
1answer
73 views

Better in ROC AUC vs. better in PR AUC

I'm comparing two classification models by computing the area under ROC and Precision-Recall curves. However sometimes one model is better with AU-ROC but worse in AU-PR, and other times it's better ...
0
votes
1answer
39 views

What is AUC of PR-curve?

I understand that AUC under ROC curve is a classic evaluation measurement for classifiers (which is basically the accuracy). However, when data is imbalanced, PR will be alternative. So, what does the ...
1
vote
2answers
75 views

Help with understanding statistical measures and Receiver Operating Characteristics

In my machine learning class we just went over statistical measures and plots. We looked at the definitions of True Positive Rate (sensitivity/recall, etc), 1-False Positive Rate (speciicity), ...
1
vote
1answer
48 views

Why AUC-PR increases when the number of positives increase?

I asked a question earlier about comparing models using Precision-Recall AUC. One of the answers included the following statement: "The larger the fraction of positives in the data set, the larger the ...
1
vote
1answer
49 views

Ranking two models based on ROC-AUC and PR-AUC

I have two methods/classifiers (completely different models) that I need to decide which one is better. The dataset is imbalanced. I trained both classifiers on the same dataset and then I computed ...
2
votes
1answer
45 views

Is it correct to use Precision-Recall AUC in a balanced dataset situation?

I have a binary classification scenario with a dataset that is unbalanced (much more negatives than positives). When I train a classifier on this dataset I get a Precision-Recall AUC of 0.7. Then I ...
0
votes
0answers
46 views

How to calculate the area under the precision-recall curve for the random classifier?

I know that the random classifier score in ROC AUC (Area under the curve) is always 0.5. My question is: how to calculate the Area under the precision-recall curve for the random classifier?
0
votes
1answer
75 views

What is the convex hull in ROC curve?

I'm reading a paper about ROC and PR curves. They mentioned the ROC convex hull but they don't define it or say what it is. Can someone please tell me the meaning of it? What is a convex hull in ROC ...
2
votes
2answers
68 views

An intuitive meaning of the area under the PR curve?

Wikipedia says that an interpretation of the area under the ROC curve is: "the area under the curve is equal to the probability that a classifier will rank a randomly chosen positive instance higher ...
1
vote
1answer
184 views

Finding true positive / negative and false positive / negative rates using R

I have a data frame with two classes. I want to find the true positive and false positive rate and then plot the ROC curve. I tried this: ...
0
votes
0answers
8 views

Tree classifier or nested model?

I am new to statistics and was wondering what the right kind of model to use for the following scenario. I have two sets of continuous observations A and B for 50 samples. These 50 samples are ...
0
votes
2answers
46 views

How to turn my data into a ROC curve in R? [closed]

I'd like to generate ROC curves in R, but I'm confused about what input to give either ROCR or pROC. I have 5000 data-points for which I know the true classification (1 or 0), and a continuous ...
0
votes
0answers
66 views

Complete Logistic Regression framework using K-Cross validation

I'm implementing a logistic regression model in a low event rate data. I have gone through many webpages (including stackoverflow, including my questions) but none answer or describe the end-to-end ...
0
votes
0answers
51 views

Significance of a dichotomized variable from a continuous variable

I am analyzing a X continuous independent variable with a Y binary response. The investigator has interested on dichotomize the X variable by the “best” cutpoint from the ROC curve (for example the ...
0
votes
2answers
82 views

ROC (Epi library) how to calculate TP, FN,TN, FP

I'm trying to find how to compute the true negative (TN), false negative (FN), true positive (TP), and false positive (FP) if I have a cutpoint like in the following picture: ...
0
votes
2answers
37 views

Use of data from ROC curve

In order to find an optimal time for initiation of treatment post surgery (oncologic patients) I created a ROC curve with death defined as event. The AUC was not significant. However, I decided to use ...
0
votes
0answers
30 views

Is it logical to stand on the chance-line (50-50 %) when we don't know a-priori probabilities?

Let we have two hypotheses $H_0$ and $H_1$ and we don't know their a-priori probabilities. If we wish to calculate the average probability of error, does it makes sense to assume 50-50 % chance of ...
0
votes
0answers
41 views

How to evaluate a trained model using parameters other than AUC in RapidMiner?

I am using RapidMiner to build predictive models trained and cross-validated by a set of medical data(65 cases. 18 attributes), I am now running trials by trying different combinations of learners and ...
0
votes
0answers
53 views

Cutoff and precision values of a binary classifier

Let's say I have fitted a binary classifier to some data and I'm varying the cutoff value, effectively producing a ROC-curve. Knowing the true proportions of positives and negatives, I can calculate ...
2
votes
2answers
102 views

ROC for more than 2 outcome categories

How do you construct ROC Curves when there are more than two outcome categories (in my case, I have four)? I've heard you should do this for the most popular group. Are there any other ideas? Are ...
0
votes
0answers
37 views

Selecting individuals from a population using a binary classifier

I have a dataset consisting of around 200 individuals, whose outcome is either of state $0$ or $1$. I am able to make binary classifiers and predictors on this set and build ROC-curves for them just ...
1
vote
3answers
136 views

ROC graph shape

Could you explain to me how the shape of a ROC curve is determined? From the following illustration, it seems that for every time the actual class (C) is positive, it goes up and when it's negative, ...
2
votes
0answers
137 views

Unbalanced dataset - ROC curve to compare classifiers?

I use the machine learning software WEKA for data mining on biological data. I would describe my dataset as unbalanced: It comprises around 2000 instances, ...
2
votes
1answer
123 views

Is it reasonable for a classifier to obtain a high AUC and a low MCC? Or the opposite?

Let's say I have 2 models: 1) High Matthew's correlation coefficient (MCC) score, low area under the curve (AUC) 2) Low MCC, high AUC When I say high and low, I mean relatively to the other model. ...
-1
votes
1answer
31 views

Which is better ROC curve or Confusion matrix?

anyone know about the predictive model evaluation? I'm confused about the ROC curve and confusion matrix. The area under the curve for ROC is represent about the accuracy of the classifier. But what ...
1
vote
2answers
539 views

Using the caret package is it possible to obtain confusion matrices for specific threshold values?

I've obtained a logistic regression model (via train) for a binary response, and I've obtained the logistic confusion matrix via ...
0
votes
0answers
75 views

Multi-class AUC in Matlab

I would like to compute the area under the ROC-courve (AUC) metric for a classifier with multiple classes. Do you know (reliable) functions for Matlab that implement methods for that, like e.g. in ...
2
votes
1answer
115 views

Differences in AUC calculation between pROC and ROCR

Does anyone know the difference in calculation between these two AUC packages? They get different results when I add in positives with predicted value of 0 (simulating a prob model where many outputs ...
3
votes
1answer
165 views

ROC-AUC and Precision-Recall for random classifiers in class imbalanced problems

I have always always understood the diagonal of the ROC plot to represent the performance of a "random" classifier (corresponding to an AUC of 0.5). Is this still the case for highly imbalanced ...
2
votes
2answers
221 views

ROC vs. Accuracy [duplicate]

If you want to compare two learning algorithms, which metric is better to use in general: ROC or accuracy? I understand that in ROC, you get both the sensitivity and specificity?
1
vote
1answer
64 views

ROC/AUC Confidence Interval

For a single ROC curve (with relevant AUC score), how can you calculate the confidence interval? (The data used to generate this ROC/AUC is available) Given my relatively limited background in this ...
0
votes
1answer
54 views

How can I get cut-off point in multivariated ROC analysis

If I have 1 independent variable (continues) and 1 dependent variable (binary), I can conduct logistic regression and ROC analysis, and I can get a cut-off point of independent variable using ROC ...
2
votes
1answer
344 views

Sample size calculation for ROC/AUC analysis

As a background, I am not familiar with stats except on a basic level. I have been tasked with doing some analysis that is out of my comfort zone. I am trying to figure out how to compute necessary ...
0
votes
0answers
24 views

Computing baseline probability

What is the meaning of the term Baseline probability in an experiment? How is it computed, say for a binary classifier? How to measure the performance of a classifier according to a given Baseline ...
2
votes
2answers
645 views

How we can draw an ROC curve for decision trees?

Normally we cannot draw an ROC curve for the discrete classifiers like decision trees. Am I right? Is there any way to draw an ROC curve for Dtrees?
2
votes
1answer
383 views

Understanding ROC curve[edited]

I'm having trouble understanding ROC curve. Is there any advantage/improvement in area under the ROC curve if I build different models from each unique subset of train set and use it to produce ...