Receiver Operating Characteristic, also known as ROC curve.
1
vote
0answers
13 views
Using AUC to compare logistic lasso and elastic net
I've seen this question answered here but I do not understand the answer. Harrell recommends using deviance based measures. David Hand (referenced in the thread) says that that the AUC is ...
2
votes
1answer
37 views
Computing by hand the optimal threshold value for a biomarker using the Youden Index
I have an empirical estimate of a ROC curve, that is, a plot of the sensitivity versus 1-specificity over all possible cut-off values of the marker.
Based on an empirical ROC curve, I would like to ...
0
votes
0answers
20 views
Obtaining threshold values from a ROC curve (R, ROCR) [migrated]
I have some models, using ROCR on a vector of the predicted class percentages, I have a performance object. Plotting the performance object with the specifications "tpr","fpr" gives me a ROC curve.
...
1
vote
1answer
42 views
Why is there a sharp elbow in my ROC curves?
I have some EEG data sets that I am testing against two classes. I can get a decent error rate from LDA (the class-conditional distributions aren't Gaussian, but have similar tails and good enough ...
1
vote
0answers
59 views
Precision, Recall and area under ROC curve as sample size increases
The following is a question from an exam paper on evaluating the performance of search engines. To this day I looked in my text book and literally close to 50 web pages and I can't
find one convincing ...
0
votes
0answers
57 views
Setting a decision threshold in one-class classification with LIBSVM
I want to do anomaly detection with the one-class SVM implementation in LIBSVM. However, the prediction output of the trained SVM is binary. In my understanding, there has to be a "radius" parameter ...
1
vote
1answer
68 views
ROC curve crossing the diagonal
I am running a binary classifier at the moment. When I plot the ROC curve I get a good lift at the beginning then it changes direction and crosses the diagonal then of course back up, making the curve ...
1
vote
1answer
33 views
Does ROC assessment of a binomial classifier serve as a good performance measure given equal weights between positives and negatives?
My problem is that i have created four candidate models that I am comparing mainly via the following performance measures: F-measure, ...
0
votes
0answers
46 views
calculating false positive rates from specificity for roc curves
I've got to create roc curves and have sensitivity and specificty values from 0-100. If i want use the false positive rate for my x axis on a scale of 0-100, do i calculate 100 minusthe value i have ...
2
votes
1answer
162 views
What exactly the ROC curve can tell us or can be inferred?
(I post this originally at http://stackoverflow.com/questions/15477282/what-exactly-the-roc-curve-can-tell-us-or-can-be-inferred, but people directed me to here. Sorry about posting this twice.)
I ...
0
votes
1answer
166 views
ROC curve and confusion matrix in classifier performance evaluation
I applied two different classifiers against the same validation set. It turns out that classifier A is better than classifier B in terms of ROC curve. However, classifier B is better than classifier ...
0
votes
0answers
49 views
Confusion with sensitivity and specificity within the survivalROC R package
I have a question about sensitivty/specificity in the survivalROC package.
I've been able to successfully use the survivalROC package to draw some ROC curves. As you know ROC curves have ...
6
votes
1answer
458 views
What is the difference in what AIC and c-statistic (AUC) actually measure for model fit?
Akaike Information Criterion (AIC) and the c-statistic (area under ROC curve) are two measures of model fit for logistic regression. I am having trouble explaining what is going on when the results of ...
2
votes
1answer
205 views
Precision - Recall: Graphical Representation
I'm a little bit confused with precision recall. I read some papers about recommender systems, where in the one paper they have a graphical representation and in the other papers they don't (they just ...
1
vote
0answers
78 views
ROC50 score - how is it calculated?
I have never found a good explanation of what the ROC50 score is and how it is calculated.
To take an example definition found on the net:
The ROC50 score is the area under the ROC curve, up to ...
0
votes
1answer
70 views
ROC curves : using package pROC : DUMMY EXAMPLE
I have been reading a lot about ROC curves, but honestly I still don't have it cleared in my mind. So I request anyone here who could explain me about it with respect to my dummy example below.
So ...
4
votes
2answers
117 views
Summary statistics of the precision-recall curve
From what I understand, one can use the AUC of the ROC curve as a summary statistic of the full curve.
Q1. Are there any similar summary statistics that one can use on a single precision-recall ...
3
votes
1answer
54 views
CAP and ROC measures
Why is the area under the curve measure better than the accuracy ratio?
I know that the AR lies between 0,5 and 1 and the relationship
$AUC=\frac{1}{2}(AR+1)$ holds, so the AUC lies between 0,75 and ...
1
vote
1answer
515 views
How to calculate a ROC curve? What kind of inputs do you need for that?
I have a microarray data which I ran through a continuous covariate (say "X"). I did this using 4 different methods.
For the results obtained from each of the 4 methods, I have the following from ...
2
votes
1answer
109 views
Analogues of sensitivity and specificity for continuous outcomes
How can I calculate the sensitivity and specificity (or analogous measures) of a continuous diagnostic test in predicting a continuous outcome (e.g., blood pressure) without dichotomizing the outcome? ...
1
vote
0answers
39 views
Computing precision of continuous classifier with ordinal outcome
I have data where a rater predicts quality on a continuous scale, and quality is then validated in buckets. For example, one rater might give scores of 0.2, 0.4, 0.7, and 0.6 for cases with outcomes ...
0
votes
0answers
110 views
Time-dependent ROC in R
I have a medical background, with absolutely no knowledge about programming. I am working on survival analysis. I have done Cox multivariate analysis on my data using SPSS. I have performed ...
1
vote
2answers
83 views
How to generate ROC Plot for semi-supervised algorithm?
By having a data-set 1000 (900 unlabeled, 100 labeled) record data-set for binary classification, I want to apply a semi supervised algorithm.
The problem is that I don't know how to get values for ...
0
votes
1answer
97 views
2
votes
1answer
111 views
Replicating a plot from the ROCR Website
I'm trying to reproduce this plot from the ROCR website:
I can get something similar as follows:
...
0
votes
0answers
124 views
comparing KNN with SVM on ROC chart
1) I wish to compare the performance of KNN and SVM on ROC chart.
a)For KNN, I obtained a linear line on ROC Chart instead of curved one.From x=0 until x=0.275, y= 0.65 and from x=0.35 until x= 0.95, ...
2
votes
2answers
224 views
Fastest way to compare ROC curves
I have a set of true positive (TP) values which are used to train a model.
I am using 5-fold cross validation to train my model (i.e. split my true positives into 5, use 4/5ths for training and ...
1
vote
0answers
168 views
Help on Regression Analysis (ROC, multinomial)
I am currently looking at continuous data and seeking a correlation with the presence (and severity, ie, normal, mild, moderate, severe) of a disease entity. Now, so far, running Spearman on SPSS, ...
4
votes
1answer
177 views
Evaluation of classifiers: learning curves vs ROC curves
I would like to compare 2 different classifiers for a multiclass text classification problem that use large training datasets. I am doubting whether I should use ROC curves or learning curves to ...
2
votes
1answer
206 views
Decision threshold for a 3-class Naive Bayes ROC curve
I have some doubts regarding how a ROC curve for a 3-class classifier (Naive Bayes) can be built.
Basically, given some test data, the classifier outputs the probabilities for each of the 3 possible ...
2
votes
2answers
576 views
Creating ROC curve for multi-level logistic regression model in R
I used the functions from this link for creating ROC curve for logistic regression model. Since the object produced by glmer in ...
0
votes
0answers
142 views
Rank Correlation Statistics Comparison
I am trying to understand the relative behavior of the following rank correlation statistics:
Spearman coefficient
Kendall Tau / Concordance percentage
Normalized Gini coefficient (area under curve ...
0
votes
0answers
30 views
Data mining, Recall and ROC curve [duplicate]
Possible Duplicate:
Recall and AUC of a binary classifier
Can anyone explain this please?
A binary (i.e., two-class) classifier may very well have a recall of 0.0 for one of the classes and ...
2
votes
1answer
181 views
Recall and AUC of a binary classifier
Is it possible for a binary classifier to have a recall of 0.0 for one of the classes and
at the same time an area under the ROC curve (AUC) of ...
1
vote
0answers
62 views
How to modify RankBoost to maximize area under recall-precision curve instead of AUC?
Using the WeakLearn algorithm from the original RankBoost paper, how do you set the optimal threshold to maximize AU-RPC (instead of AUC)? And, once that threshold is set, how do you calculate the ...
3
votes
2answers
1k views
ROC curve for discrete classifiers like SVM: Why do we still call it a “curve”?, Isn't it just a “point”?
In the discussion : how to generate a roc curve for binary classification, I think that the confusion was that a "binary classifier" (which is any classifier that separates 2 classes) was for Yang ...
2
votes
1answer
97 views
Predictive Probabilities
Other than a calibration plot, is there a way to decide how good one models' predictive probabilities as compared to another model.
I'm not interested in error rates as I find them ineffective for ...
4
votes
1answer
92 views
Why do you maximize specificity for circumstantial findings?
Tests for a medical condition can be classified into two categories:
Tests on an asymptomatic (no reason to suspect condition) patient where the result is a circumstantial finding; and
Tests on a ...
0
votes
1answer
107 views
Estimating maximum predictive power in noisy data
When assessing model performance, one would like to separate prediction errors due to limitations of the model from those errors due to intrinsic noise. For example, in noisy data, AUC in an ROC ...
-1
votes
1answer
1k views
How to plot ROC curve with TP, FP and FN alone with different thresholds?
I want to plot a ROC for my detection algorithm which is used to detect features
in image. I had obtained true positive, false positive and false negative from
the algorithm. There is no true ...
1
vote
1answer
129 views
Comparing continuous predictors for a dichotomous variable
I have two continuous predictor variables to predict a dichotomous variable. In addition i have constructed two (interaction) models, based on domain knowledge which use both variables to predict the ...
0
votes
0answers
22 views
Multiple ROC curves for a multiple class problems [duplicate]
Possible Duplicate:
ROC curve for knn model
I am running 6 different algorithms and need to plot the ROC curve for all of them in one single plot. It is not a binary class problem, but ...
1
vote
2answers
277 views
ROC curve for knn model
I am using ROCR package and I was wondering how can one plot a ROC curve for knn model in R? Is there any way to plot it all with this package?
I don't know how to ...
2
votes
2answers
622 views
Interpreting logistic regression
I need to perform a logistic regression to to see if a group of variables which are found to be significantly associated with an outcome (by univariate tests) have significant impact on the outcome ...
2
votes
2answers
292 views
ROC vs Accuracy
I've designed a 4 classifiers which perform pretty decently (all of them are above 90% in accuracy).
However, they don't have similar AUC for their respective ROC curves (obviously, it doesn't have ...
2
votes
1answer
239 views
Evaluation of classifier using ROC curve in the presence of rare events
For binary classification problems where the data is highly imbalanced, i.e. much more negative samples than positive samples, it is recommended to evaluate the performance of a classifier using the ...
1
vote
1answer
303 views
Binary classifier - dividing dataset into training and evaluation sets
I have a Hidden Markov Model for binary classification and two datasets:
positive instances
negative instances (way more data than the positive ones)
In order to evaluate the performance of the ...
1
vote
0answers
32 views
How to calculate ROC(Ag) in this sound perception study?
I am studying individual differences in sound perception where I ask participants to listen to a pair of sounds and judge whether they are the same or different.
The answer choice and scoring are as ...
6
votes
4answers
2k views
How to determine best cutoff point and it`s confidence interval using ROC curve in R?
I have the data of a test that could be used to distinguish normal and tumor cells. According to ROC curve it looks good for this purpose (area under curve is 0.9):
My questions are:
How to ...
3
votes
1answer
426 views
Plotting overlaid ROC curves
I'm trying to make overlaid ROC curves to represent successive improvements in model performance when particular predictors are added one at a time to the model. I want one ROC curve for each of about ...
