Receiver Operating Characteristic, also known as ROC curve.

learn more… | top users | synonyms

1
vote
0answers
13 views

Visualizing multi-class ROC analysis [duplicate]

I am running a multinomial logistic regression model (with 3 possible outcomes) in R. I am trying to find the best way to assess the predictive power/accuracy of the model, and the best thing I've ...
0
votes
0answers
9 views

Differences between cross validation and bootstrapping to estimate the standard error of the AUC of a given ROC curve

I know there's been some discussion on differences between CV and bootstrapping for estimating out-of-sample prediction error of a classifier. For example, in here (Differences between cross ...
1
vote
2answers
29 views

Which data split should I use to determine cutoff point for classification?

I'm building a classification model using the caret package. I'm splitting my dataset in train and test (80/20) and training using 10-fold cross-validation repeated ...
0
votes
0answers
13 views

Relationship between Accuracy Ration and Area Under Curve in discrete model?

How can we establish connection between AR (Accuracy Ration) and AUC (Area Under Curve) when AR depends on order of data? I am talking about relation $AR = 2AUC - 1$. Here ...
0
votes
1answer
42 views

Is ROC or PR curve only the overall performance measure for classification

We can use ROC or PR curve to access the performance of the classifier,especially on imbalance data. But it is a curve with parameter threshold, even if we get a high ROC or PR performance, which ...
1
vote
3answers
84 views

Averaging ROC curves over folds in cross-validation

I have data from 10-fold cross-validation experiment: for each fold I have a predictor and a response variable so I can generate ROC curve and compute area under the ROC curve. I have a series of ...
0
votes
0answers
22 views

AUC of Survival ROC smaller than 0.5

If we obtain an AUC < 0.5 analysing a binary response variable (0/1) with a marker, this means the negative values of marker is associated to the value 1 of the response. We could use the trick to ...
0
votes
1answer
49 views

Can a Precision-Recall curve or a ROC curve be horizontal?

I am working on a binary classification task on imbalanced data. Since the accuracy is not so meaningful in this case. I use Scikit-Learn to compute the Precision-Recall curve and ROC curve in order ...
0
votes
0answers
24 views

Estimate ROC curve using binormal distribuiton

I am conducting a meta-analysis on diagnostic studies but for each study I have only mean and standard deviation reported. How can I estimate the ROC curve using the binormal assumption in R ? Tks,
0
votes
0answers
24 views

Classification Model on Single Feature?

this is my first time using StackExchange so forgive me if I commit any faux paus with this question, and it has only been a few months since I first started learning machine learning. In my ...
1
vote
0answers
19 views

Relationship between ROC (receiver operating characteristic) curve and cross-over error rate?

I do not understand this relationship. According to wikipedia, the CER can easily be obtained from the ROC curve. Equal error rate or crossover error rate (EER or CER): the rate at which both ...
0
votes
0answers
17 views

pearson correlation vs AUC of ROC

is it possible to get not significant correlation with p>0.05, but the area under ROC curve is excellent with score 0.99?
0
votes
0answers
18 views

What does it mean if the ROC scores are quite different when using the Stratified K fold with and without shuffling?

I'm currently building a random forest classification and trying to measure the model performance by the [mean ROC area]. With the same data set: When I use cross_validation.StratifiedKFold(y, ...
0
votes
3answers
33 views

Using roc() in R with a parametric response variable name in the formula [closed]

I have a column named inp, and 10 columns named resp1, ..., resp10 in a matrix in R, and I want to compute receiver operating ...
0
votes
1answer
40 views

plot ROC curve from glm model using gaussian model

I have some data (322 x 4) that looks like that ...
1
vote
2answers
65 views

what is the difference between area under roc and weighted area under roc?

Thanks in advance for the help. I have an unbalanced dataset that I am using for a binary classification problem. The classes are unbalanced. I believe that in such a case that weighted area under ...
1
vote
1answer
50 views

Area under ROC given the two distributions

I have two distributions p1 and p2 (generating in R using distr package), and I want to compute the area under the ROC curve. To construct the ROC curve, I have to compute the probability of detection ...
1
vote
2answers
73 views

Relationship between ROC curve and statistical significance in comparison of two groups

My question is about any possible relationship between the significance of a comparison test (Mann-Whitney U) and the ROC curve. If the comparison test is strongly significant, should we expect a ...
0
votes
1answer
43 views

Did I understand ROC curves correctly?

I want to classify objects by their area into two classes. I implemented several area estimates that I want to compare. For each object, I have a gold standard indicating to which of the two classes ...
3
votes
1answer
63 views

ROC curve for two-sided cut-off

I am very, very confused about ROC curves. I have a Bayesian model which outputs a prevalence on a continuous scale between 0 and 1. I have a classification I would like to use that classifies that ...
1
vote
0answers
33 views

Formula to derive probabilities when using 'gbm' method through 'caret' package in R

I am training a classification problem by 'gbm' algorithm through 'caret' package in r.The response variable is a yes/nope type. Here 'objmodel' is the model I trained through method='gbm' and package ...
0
votes
0answers
14 views

What type of curve to use in object detection task to measure performance of detector?

I have an object detection task with one object type(one object type + background = object detection with sliding window as binary classification of each window). And my data is unbalanced(many ...
0
votes
1answer
77 views

The relation between a confusion matrix and a ROC curve

As an example I have a confusion matrix that shows good accuracy but poor performance on sensitivity because of imbalanced classes. I made this fictive table for a presentation. ...
0
votes
1answer
40 views

Calculating AUC for a GEE

I have used the geeglm package to build a GEE that predicts animal activity (a binary response, active or not) from weather data (e.g., Temperature, a continuous variable). TEMPC <- ...
0
votes
0answers
40 views

What is the difference between GINI and AUC curve interpretation?

we used to create GINI curve using lift created with help of percentage of good and bad for scorecard modelling. But what I have studied that ROC curve is created using Confusion matrix with ...
1
vote
1answer
26 views

compute ROC from Sensitivity and Specificity

It might sound like a stupid question, but I've got to ask it. Let's say I have a classifier, with one discrete bounded parameter. I have run the classifier over all possible parameter values, and ...
0
votes
0answers
11 views

ROC Analyses - can collapse assessment time periods?

I hope you can provide some advice on doing ROC analyses. I have been running ROC analyses on several different screening tests and have a question about one of my data sets. It is a longitudinal data ...
2
votes
1answer
26 views

ROC and constant factor on probabilities

I play around with a few data to learn and I am wondering about something; I can evaluate my results with ROC which is processed from FP and FN. I had predicted a few probabilities for my events to ...
0
votes
0answers
61 views

ROC curve cut off and weights

I have a dependent variable distinguishing between patients that should go to treatment A or treatment B. I want to develop a questionnaire containing binary variables that should decide if the ...
0
votes
0answers
10 views

Is there any lower limit for number of positives when generating lift plot?

I am wondering if there is any condition on number of positives in test set when I am trying to compute lift plot to check the properties of my classifier?
9
votes
2answers
500 views

Difference between regression analysis and curve fitting

Can anybody please explain to me the real difference(s) between regression analysis and curve fitting (linear and nonlinear), with an example if possible? It seems that both try to find a ...
0
votes
0answers
37 views

Interpretation and meaningfulness of regression coefficients

I have performed logistic regression on banking data which is trying to predict the bad customers correctly due to the cost involved. I have build a model and pasting a picture of the output obtained ...
0
votes
0answers
16 views

ROC and post estimation COX Harrell's C using your dataset

I have built a predictive model using a combination of logistic and cox regression models. I did it using a dataset of about 5000 records. I would like to calculate the AUC and the Harrell's post ...
2
votes
1answer
46 views

So many significant explanatory variables and so small auc

Have you ever seen a model with almost every significant variable and such small auc (area under the ROC curve) ? What might be the cause of it? When I saw summary of a model I thought this model will ...
0
votes
0answers
38 views

How is the mean area under the curve calculated?

I am using 10-fold cross-validation for performance estimation. From each of the ten iterations, I get an area under the curve (AUC) metric, e.g. ...
0
votes
0answers
57 views

Area under ROC curve vs. Accuracy in unbalanced sample

I have a binary classification problem with 3000 samples (number of 1 as outputs = 300, number of 0 as outputs = 1700). After balancing database (selecting 300 samples from 0 outputs) I trained the ...
1
vote
1answer
43 views

Constructing an ROC curve without true negatives?

I’m not really a statistician by training so please excuse any incorrect terminology or if this is a painfully obvious question. I am trying to assess the performance of two algorithms. I don’t wish ...
3
votes
1answer
79 views

Simple Question about ROC Curve

Motivated by this reference, it states under ROC Space When evaluating a binary classifier, we often use a Confusion Matrix...however here we need only TPR and FPR I'd feel more comfortable if ...
4
votes
1answer
199 views

Different shapes of an ROC curve

What are the possible shapes of an ROC curve? Is it necessary for an ROC curve to be shaped like a normal distribution curve? Can we regard the following two curves as ROC with the area under the ...
0
votes
0answers
28 views

Forward sequential feature selection improving classifier performance?

I was in a bit of a conversation with a co-worker about using forward selection. My training data is on order of ~6,000 w/ dimensionality of 1,200, and testing data on order of ~3,000. Currently, I'm ...
3
votes
1answer
61 views

Getting the optimal age cut off

I am working with a database of diseased and non-diseased patients. I would like to recommend an optimal age cut off for disease screening. The common problem with screening is that it creates a ...
0
votes
0answers
28 views

ROC for optimal cut-off using Svyglm object

I am trying to estimate optimal cut-points via ROC for a complex survey data. I able to achieve this task with ROC(in Epi) and OptimalCutpoints package but for the unweighted sample. The ...
4
votes
1answer
106 views

Model comparison between glm (with Firth correction), random Forest, penalised SVM

I am currently developing three models to classify features of gene sites. I was using glm (with Firth correction), random Forest and SVM to build the models and I used forward and backward ...
1
vote
2answers
76 views

Using ROC curve for balanced data

I understand that using the area under the ROC curve is a useful error measurement for unbalanced data. What happens if we use it for balanced data?
1
vote
0answers
190 views

How can I calculate AUC using Gini coefficient?

In the Gini Coefficient's Wikipedia page, it is defined as $G= 1 - \frac{\Sigma_{i=1}^n f(y_i)(S_{i-1}+S_i)}{S_n}$ for discrete variables, where $S_i= \Sigma_{j=1}^i f(y_i)y_i$ and $S_0=0$ ($y$ being ...
1
vote
0answers
135 views

Calculating two-tailed p-value from z-score for ROC AUC comparison

I am comparing two predictive models by their bootstrapped ROC AUCs with the method originally described by Hanley and McNeil and modified for bootstrapped data by Robin et al. I'm calculating the ...
1
vote
1answer
53 views

ROC curve when the numbers of both total positives and negatives are unknown

I have to use a roc curve to find an agreement between sensitivity between the FPR and TPR. The problem is that I only know the true positives. How could I circumvent this? More accurate, I have a ...
0
votes
1answer
48 views

How to know if my data is balanced or imbalanced for an ROC curve analysis?

I am doing a research on the reliability of different models in detecting hidden defects in a test specimen. I have made a test specimen with defect prevalence about 25% (12 positive out of 49 total ...
0
votes
1answer
71 views

What does ROC-EER in percent stand for?

Ive tried to understand what the ROC Curve represents and what EER (Equal Error Rate) means. And I somehow think I got to understand some of the explanations I read on the internet and videos I ...
6
votes
2answers
1k views

d prime with 100% hit rate probability and 0% false alarm probability

I would like to calculate d prime for a memory task that involves detecting old and new items. The problem I have is that some of the subjects have hit rate of 1 and/or false alarm rate of 0, which ...