Receiver Operating Characteristic, also known as ROC curve.

learn more… | top users | synonyms

0
votes
0answers
20 views

ROC curve and its function beginner

I have 3 features of a signal (example: amplitude, frequency, energy). I want to check which feature is the best to represent that particular signal. That signal is classified into two categories ...
0
votes
1answer
53 views

I want to learn about ROC curve — what is the canonical textbook?

I want to learn about Receiver-Operator-Characteristic curves, and metrics. I have read through online webpages with some basics, and I have used MATLAB built-ins to create ROC plots. It tells me ...
0
votes
0answers
24 views

How do the ROC cutoffs relate to predictors?

Apologies for this rather simple question, but I haven't been able to find a definition online. What does the ROC cutoffs represent for the AUC package? Specifically, how does it relate to the ...
2
votes
3answers
97 views

The value of adding the ROC graph if the AUC is given

I always see in papers that when they want to show how they classifiers performed, they provide ROC graph and the AUC score. Now as far as I know only the AUC tells how well the classifier performed, ...
3
votes
0answers
95 views

How to choose the cutoff probability for a rare event Logistic Regression

I have 100,000 observations (9 dummy indicator variables) with 1000 positives. Logistic Regression should work fine in this case but the cutoff probability puzzles me. In common literature, we ...
1
vote
2answers
55 views

Area under the ROC curve or area under the PR curve for imbalanced data?

I have some doubts about which performance measure to use, area under the ROC curve (TPR as a function of FPR) or area under the precision-recall curve (precision as a function of recall). My data is ...
5
votes
3answers
240 views

Why is AUC higher for a classifier that is less accurate than for one that is more accurate?

I have two classifiers A: naive Bayesian network B: tree (singly-connected) Bayesian network In terms of accuracy and other measures, A performs comparatively worse than B. However, when I use the ...
3
votes
1answer
55 views

Understanding random forest, gini, and KS

I'm a beginner machine learning user, doing my first predictive model using random forest. I have some questions regarding the way to measure how good a model is (Gini area from roc curve, and KS), ...
4
votes
2answers
89 views

How to do external validation of logistic regression models and perform model benchmarking

Quality assessment in trauma has for > 25 years been done with the US derived logistic regression model, the TRISS model. DV: survival/death and IVs: physiologic derangement (continuous), anatomic ...
0
votes
0answers
40 views

Random forest highly imbalanced dataset: how to test _ create ROC curve?

I have a dataset containing 10,000 examples with 8 features. I would like to create a random forest to classify this dataset in two classes, more specific: substrate / no substrate. In this dataset ...
0
votes
1answer
68 views

Why use d-prime instead of percent correct?

In signal detection theory, people often use $d'$ to assess performance. Apart from the fact that $d'$ is in $z$ units (units of measurement transformed to standard deviation units, i.e., $z$ scores), ...
2
votes
2answers
186 views

How to create ROC curve to assess the performance of regression models?

I knew that, ROC curve are use to assess the performance of classifiers. But is it possible to generate ROC curve for the regression model? If yes, How?
2
votes
1answer
87 views

Statistics for Area under the ROC curve

I have a question regarding statistical evaluation of the AUC. In their paper (http://www.jstor.org/stable/2531595), DeLong et al. describe a method to evaluate AUC curves. (Another good explanation ...
0
votes
1answer
40 views

LIBSVM ROC PLOt [closed]

my question is "What is command used to generate ROC PLOT in libsvm as well as for specificity , sensitivity and precision what is the command in libsvm " i Am using ubuntu
0
votes
0answers
18 views

Measuring the performance of a classifier that estimates bin counts for a Poisson process

I have some data that results from the convolution of a Poisson random process with a decaying exponential, plus some additive Gaussian noise. I'm working on designing a classifier that returns a ...
0
votes
0answers
15 views

VUS (volume under the surface)

After some research I did with the VUS(volume under the surface) for 3 or more class ordinal regression,I have this question. Let Y be a continuous out come from a 3 or more classifier. For example ...
3
votes
2answers
68 views

ROC graph interpretation

I'm reading Fawcett's 2004 paper on ROC graphs for machine learning algorithms, which can be found here. On page 7-8 he shows a simple ROC example and makes some interpretations that I don't ...
1
vote
2answers
96 views

Bootstrap to evaluate variance of AUC ROC

I have a toy dataset and want to eval the AUC - ROC with bootstrap ...
2
votes
1answer
38 views

Using predict_proba with sklearn's multiclass SVC

I'm using python's sklearn for multi-class classification (SVC) When using the predict method, i get very high scores with my dataset, However, I want to plot ROC curves for each of my classes. That ...
2
votes
0answers
52 views

Does it make sense to plot train and test results on a common ROC graph?

I am building a model using SPSS with 80% of my data. I can plot a ROC curve for both the training set and separately for the test set. I can´t find a way to plot both on the same graph. Since both ...
1
vote
0answers
42 views

Leave one subject out: Evaluating classifier performance

I have a small dataset, and I am trying to train 1-vs-all classifiers (class X or not class X) using a linear SVM. The dataset is simply too small to split up 60/40, 80/20 or anything like that. Also, ...
0
votes
0answers
52 views

How to compare several learning algorithms in a single domain

I want to compare several learning algorithms in a single domain. In the book " Evaluating Learning Algorithms" I saw the procedures to compare two learning algorithms in a single domain or multiple ...
0
votes
1answer
17 views

How to compute for the verification rate at 0.001 FAR?

I want to test my pair matching method based on this evaluation metric but unsure how to do this? I only have positive (matching) test samples.
1
vote
0answers
32 views

Calculate AUC of a logistic regression model [duplicate]

I have a data sample of a bank loan history of customers. I have performed logistic regression testing on the sample for finding out how the loan repayment(YES/NO) is dependent on various factors. I ...
1
vote
1answer
38 views

which performance metrics to classify model

I wonder between two performance metrics for classification models: accuracy and area under ROC curve (AUC), which one is to be preferred in which conditions? examples appreciated
0
votes
1answer
64 views

Accuracy and area under ROC curve (AUC)

If we group examples with and without class labels using clustering techniques by treating the class as an ordinary nominal attribute, the resulting clusters can then be used for classifying test ...
2
votes
3answers
154 views

Methods to determine cut-off between groups

I am working with a dataset of diseased vs non-diseased patients. I'm looking for a cut-off value for determining the status of a new patient. This is intended for lab technicians and MDs to allow for ...
2
votes
2answers
139 views

Memory problems using pROC library in R to estimate confidence interval for area under the curve

I am trying to use the R pROC package to estimate the confidence interval for the area under the curve (AUC). I am running into a memory limitation. My R script looks like the following: ...
1
vote
1answer
85 views

Area Under ROC Curve for Multiple Classes

I am working with a highly class-skewed three class classification problem. The class percentages are A = 1.8%, B = 17.5% and C = 80.7%. According to this paper, the following definition of ...
1
vote
3answers
75 views

Is this classifier random?

I trained a classifier and got the following stats on some test data: ...
4
votes
3answers
312 views

In R how to compute the p-value for area under ROC

I struggle to find a way to compute the p-value for the area under a receiver operator characteristic (ROC). I have a continuous variable and a diagnostic test result. I want to see if AUROC is ...
3
votes
3answers
148 views

pattern of ROC curve and choice of AUC

I am using ROC curves and full AUC values to compare different models, using simulated data. Now I think I am confused with the interpretations of ROC curves and AUC values. Please see the figure ...
3
votes
2answers
111 views

Estimating ROC/AUC on large data sets?

Plotting an ROC curve of a classifier compared to cases requires that the data set be sorted first on the classifier score. I am in a position where I need to calculate ROC on a large data set very ...
0
votes
1answer
142 views

ROC curves and AUC in simulations to compare models

I am using ROC curves to compare different methods but not sure if I need to re-simulate datasets using different seeds in R in order to reduce the "by-chance" issue for a particular output. Here is a ...
0
votes
0answers
258 views

Finding Optimal threshold from ROC curve (SVM classifier in Matlab)

I plotted the ROC curve for the SVM classifier performance in Matlab. I would need to find the optimal threshold value from the ROC curve and need to set the SVM at that particular threshold for ...
2
votes
1answer
99 views

How to draw ROC curve with three response variable?

I have a toy example reproduced below in which the response variable has three possible classes. I am trying to create an ROC but not sure how to deal with it when there are three classes. Any help ...
6
votes
1answer
257 views

Evaluating a logistic regression

Recently I’ve been working on a logistic model and I´m having some difficulties evaluating the results. I am still learning all this, so I apologise in advance for the mistakes. I would still ...
2
votes
1answer
75 views

Cutoff on ROC curve

@cbeleites mentions in response to the question How to determine best cutoff point and its confidence interval using ROC curve in R? that if one could specify the relationship between a false ...
1
vote
1answer
105 views

Different score range when calculating area of under curve in ROC curves

I have two classifiers which try to classify the same data sets. In order to check the efficiency of the classifiers I intend to plot the curves and calculate the AUC value. The concern is that one of ...
4
votes
3answers
748 views

Area under curve of ROC vs. overall accuracy

I am a little bit confusing about the Area Under Curve (AUC) of ROC and the overall accuracy. Will the AUC be proportional to the overall accuracy? In other words, when we have a larger overall ...
1
vote
0answers
57 views

Differentiation and dimensionality reduction

I have a matrix $x=N\times M$ of $N$ data points, where each one has $M$ features. Also, $y$ is the binary labels vector. In my case, $N$ is much smaller than $M$, so before running a classifier like ...
1
vote
3answers
1k views

ROC and multiROC analysis: how to calculate optimal cutpoint?

I'm trying to understand how to compute the optimal cut-point for a ROC curve (the value at which the sensitivity and specificity are maximized). I'm using the dataset ...
5
votes
1answer
264 views

Why does my ROC curve look like this (is it correct?)

I have a ROC curve generated for a multivariate logistic regression. Does it look correct? This is what I've done: Solve $\theta_0 + \theta_1X_1 + \theta_2X_2 ... = Y$ for the $\theta$s Iterate ...
1
vote
0answers
91 views

Validating logistic regression - formulas needed for simulation

This is related to a data classification problem having a Boolean output variable. Summary: Once I perform the ML task using Logistic regression, I get the required coefficients. I use the ...
1
vote
1answer
282 views

Calculation of AUC value from ROC Curve

Is there any tool that can calculate the AUC value from a ROC curve if I already know how many samples are true positive, true negative, false positive, false negative out of 500 samples? Specificity ...
0
votes
0answers
47 views

Tool for calculating AUC Value [duplicate]

Is there any tool that can calculate the AUC value from a ROC curve if I already have true positive, true negative, false positive, false negative values.
2
votes
3answers
158 views

How can I tell if my binary classifier is any good?

Say I have a data set with 10,000 rows and the target is a binary variable with 1500 positives (1's) and 8500 negatives (0's). I run a model and get predictions on the 0-1 interval. My question is ...
1
vote
2answers
118 views

Effective validity of AUROC as performance measure: what about “very high” AUROC values?

The Area Under ROC curve (AUROC) is a quantity used to quantify performance of classifiers. I am currently interested in the most basic drawbacks of using AUROC as unique performance measure. I got ...
2
votes
2answers
469 views

Using principal components in a linear discriminant analysis for a diagnostic test

I am interested in building a linear discriminant function to discriminate between 2 groups, out of 60 variables. (I'm planning to select the most discriminative of the variables for a future ...
2
votes
2answers
121 views

Alternating AUC curve. What does it mean?

Why do I see my ROC curve crossing the line from (0,0) to (1,1) (i.e. the 0-1 line)? I have the following test data as a tab-separated testdata.txt file. Running my R code (given below) multiple ...