Receiver Operating Characteristic, also known as ROC curve.

learn more… | top users | synonyms

0
votes
0answers
20 views

Implementation of ROC Curve in my data?

I have successfully created 2x2 contingency table.The contingency table is created through the following process: ...
0
votes
1answer
28 views

MSE vs area under ROC

I'm testing the performance of two binary classifiers on a simulated dataset. I'm seeing that classifier 1 has a higher MSE (mean squared error / classification error) than classifier 2, but ...
0
votes
0answers
11 views

Area under the receiver operating characteristics and disease prevalence

Can anybody give me an example (with numbers) of how the area under the receiver operating characteristics curve (AUC) does not depend on the disease prevalence?
1
vote
0answers
17 views

Open discussion in ROC curve, Confusion Matrix and Venn Diagram

In am trying to use some sort of Venn diagram in order to give a visual interpretation for a particular point in a ROC curve space. This made me think that it might be good to build a tool to explore ...
0
votes
1answer
36 views

Which standard error formula for the area under the ROC curve should I use

I am trying to assess the performance of a procedure in identifying something abnormal in the cities of a particular state in USA. I tested this procedure on six different cities with different ...
2
votes
0answers
26 views

Using the ROC curve in case-control study

I'm conducting a case-control study about vitamin D and the risk of IR (Insulin resistance) among obese individuals (Cases: obese and Controls: normal body weight). I have vitamin D level and IR level ...
2
votes
2answers
54 views

How to interpret 95% confidence interval for Area Under Curve of ROC?

(I am following this paper, from page 47 on ...
1
vote
0answers
39 views

converting continous variable to discrete while maximizing gini

I have a continuous independent variable which is used to explain the dependent binary variable in logistic regression. The model user's requirement is to group this continuous variable to 6 bins. I ...
0
votes
2answers
75 views

Why ROC increasing with more variable addition in logistic regression?

I have a dataset that contains a credit card flag for members(1/0) and certain set of characteristics. I have a final set of 60 variables which I applied in logistic regression and obtained an roc of ...
2
votes
0answers
15 views

Calculating ROC curve for two gaussian distributions with equal variance

I recall a simple formula in a paper which relates the distance between two gaussian distributions (what psychologists refer to as d') to the ROC curve under the assumption that both distributions ...
0
votes
0answers
21 views

Difference between ROC and Kappa and TSS

I am using species distribution model to model the absence-presence of species against a set of predictors. I was wondering if anyone can guide me with the following: My model gives me a high ROC ...
1
vote
0answers
13 views

Visualizing multi-class ROC analysis [duplicate]

I am running a multinomial logistic regression model (with 3 possible outcomes) in R. I am trying to find the best way to assess the predictive power/accuracy of the model, and the best thing I've ...
0
votes
1answer
34 views

Differences between cross validation and bootstrapping to estimate the standard error of the AUC of a given ROC curve

I know there's been some discussion on differences between CV and bootstrapping for estimating out-of-sample prediction error of a classifier. For example, in here (Differences between cross ...
1
vote
2answers
39 views

Which data split should I use to determine cutoff point for classification?

I'm building a classification model using the caret package. I'm splitting my dataset in train and test (80/20) and training using 10-fold cross-validation repeated ...
1
vote
0answers
18 views

Relationship between Accuracy Ration and Area Under Curve in discrete model?

How can we establish connection between AR (Accuracy Ration) and AUC (Area Under Curve) when AR depends on order of data? I am talking about relation $AR = 2AUC - 1$. Here ...
0
votes
1answer
44 views

Is ROC or PR curve only the overall performance measure for classification

We can use ROC or PR curve to access the performance of the classifier,especially on imbalance data. But it is a curve with parameter threshold, even if we get a high ROC or PR performance, which ...
1
vote
3answers
93 views

Averaging ROC curves over folds in cross-validation

I have data from 10-fold cross-validation experiment: for each fold I have a predictor and a response variable so I can generate ROC curve and compute area under the ROC curve. I have a series of ...
0
votes
0answers
31 views

AUC of Survival ROC smaller than 0.5

If we obtain an AUC < 0.5 analysing a binary response variable (0/1) with a marker, this means the negative values of marker is associated to the value 1 of the response. We could use the trick to ...
0
votes
1answer
72 views

Can a Precision-Recall curve or a ROC curve be horizontal?

I am working on a binary classification task on imbalanced data. Since the accuracy is not so meaningful in this case. I use Scikit-Learn to compute the Precision-Recall curve and ROC curve in order ...
0
votes
0answers
26 views

Estimate ROC curve using binormal distribuiton

I am conducting a meta-analysis on diagnostic studies but for each study I have only mean and standard deviation reported. How can I estimate the ROC curve using the binormal assumption in R ? Tks,
0
votes
0answers
27 views

Classification Model on Single Feature?

this is my first time using StackExchange so forgive me if I commit any faux paus with this question, and it has only been a few months since I first started learning machine learning. In my ...
1
vote
0answers
20 views

Relationship between ROC (receiver operating characteristic) curve and cross-over error rate?

I do not understand this relationship. According to wikipedia, the CER can easily be obtained from the ROC curve. Equal error rate or crossover error rate (EER or CER): the rate at which both ...
0
votes
0answers
23 views

pearson correlation vs AUC of ROC

is it possible to get not significant correlation with p>0.05, but the area under ROC curve is excellent with score 0.99?
0
votes
0answers
20 views

What does it mean if the ROC scores are quite different when using the Stratified K fold with and without shuffling?

I'm currently building a random forest classification and trying to measure the model performance by the [mean ROC area]. With the same data set: When I use cross_validation.StratifiedKFold(y, ...
0
votes
3answers
35 views

Using roc() in R with a parametric response variable name in the formula [closed]

I have a column named inp, and 10 columns named resp1, ..., resp10 in a matrix in R, and I want to compute receiver operating ...
0
votes
1answer
51 views

plot ROC curve from glm model using gaussian model

I have some data (322 x 4) that looks like that ...
1
vote
2answers
76 views

what is the difference between area under roc and weighted area under roc?

Thanks in advance for the help. I have an unbalanced dataset that I am using for a binary classification problem. The classes are unbalanced. I believe that in such a case that weighted area under ...
1
vote
1answer
52 views

Area under ROC given the two distributions

I have two distributions p1 and p2 (generating in R using distr package), and I want to compute the area under the ROC curve. To construct the ROC curve, I have to compute the probability of detection ...
1
vote
2answers
81 views

Relationship between ROC curve and statistical significance in comparison of two groups

My question is about any possible relationship between the significance of a comparison test (Mann-Whitney U) and the ROC curve. If the comparison test is strongly significant, should we expect a ...
0
votes
1answer
50 views

Did I understand ROC curves correctly?

I want to classify objects by their area into two classes. I implemented several area estimates that I want to compare. For each object, I have a gold standard indicating to which of the two classes ...
3
votes
1answer
65 views

ROC curve for two-sided cut-off

I am very, very confused about ROC curves. I have a Bayesian model which outputs a prevalence on a continuous scale between 0 and 1. I have a classification I would like to use that classifies that ...
1
vote
0answers
41 views

Formula to derive probabilities when using 'gbm' method through 'caret' package in R

I am training a classification problem by 'gbm' algorithm through 'caret' package in r.The response variable is a yes/nope type. Here 'objmodel' is the model I trained through method='gbm' and package ...
0
votes
0answers
17 views

What type of curve to use in object detection task to measure performance of detector?

I have an object detection task with one object type(one object type + background = object detection with sliding window as binary classification of each window). And my data is unbalanced(many ...
0
votes
1answer
99 views

The relation between a confusion matrix and a ROC curve

As an example I have a confusion matrix that shows good accuracy but poor performance on sensitivity because of imbalanced classes. I made this fictive table for a presentation. ...
0
votes
1answer
47 views

Calculating AUC for a GEE

I have used the geeglm package to build a GEE that predicts animal activity (a binary response, active or not) from weather data (e.g., Temperature, a continuous variable). TEMPC <- ...
0
votes
0answers
79 views

What is the difference between GINI and AUC curve interpretation?

we used to create GINI curve using lift created with help of percentage of good and bad for scorecard modelling. But what I have studied that ROC curve is created using Confusion matrix with ...
1
vote
1answer
29 views

compute ROC from Sensitivity and Specificity

It might sound like a stupid question, but I've got to ask it. Let's say I have a classifier, with one discrete bounded parameter. I have run the classifier over all possible parameter values, and ...
0
votes
0answers
11 views

ROC Analyses - can collapse assessment time periods?

I hope you can provide some advice on doing ROC analyses. I have been running ROC analyses on several different screening tests and have a question about one of my data sets. It is a longitudinal data ...
2
votes
1answer
26 views

ROC and constant factor on probabilities

I play around with a few data to learn and I am wondering about something; I can evaluate my results with ROC which is processed from FP and FN. I had predicted a few probabilities for my events to ...
0
votes
0answers
66 views

ROC curve cut off and weights

I have a dependent variable distinguishing between patients that should go to treatment A or treatment B. I want to develop a questionnaire containing binary variables that should decide if the ...
0
votes
0answers
10 views

Is there any lower limit for number of positives when generating lift plot?

I am wondering if there is any condition on number of positives in test set when I am trying to compute lift plot to check the properties of my classifier?
9
votes
2answers
564 views

Difference between regression analysis and curve fitting

Can anybody please explain to me the real difference(s) between regression analysis and curve fitting (linear and nonlinear), with an example if possible? It seems that both try to find a ...
0
votes
0answers
37 views

Interpretation and meaningfulness of regression coefficients

I have performed logistic regression on banking data which is trying to predict the bad customers correctly due to the cost involved. I have build a model and pasting a picture of the output obtained ...
0
votes
0answers
16 views

ROC and post estimation COX Harrell's C using your dataset

I have built a predictive model using a combination of logistic and cox regression models. I did it using a dataset of about 5000 records. I would like to calculate the AUC and the Harrell's post ...
2
votes
1answer
49 views

So many significant explanatory variables and so small auc

Have you ever seen a model with almost every significant variable and such small auc (area under the ROC curve) ? What might be the cause of it? When I saw summary of a model I thought this model will ...
0
votes
0answers
39 views

How is the mean area under the curve calculated?

I am using 10-fold cross-validation for performance estimation. From each of the ten iterations, I get an area under the curve (AUC) metric, e.g. ...
0
votes
0answers
63 views

Area under ROC curve vs. Accuracy in unbalanced sample

I have a binary classification problem with 3000 samples (number of 1 as outputs = 300, number of 0 as outputs = 1700). After balancing database (selecting 300 samples from 0 outputs) I trained the ...
1
vote
1answer
44 views

Constructing an ROC curve without true negatives?

I’m not really a statistician by training so please excuse any incorrect terminology or if this is a painfully obvious question. I am trying to assess the performance of two algorithms. I don’t wish ...
3
votes
1answer
87 views

Simple Question about ROC Curve

Motivated by this reference, it states under ROC Space When evaluating a binary classifier, we often use a Confusion Matrix...however here we need only TPR and FPR I'd feel more comfortable if ...
4
votes
1answer
213 views

Different shapes of an ROC curve

What are the possible shapes of an ROC curve? Is it necessary for an ROC curve to be shaped like a normal distribution curve? Can we regard the following two curves as ROC with the area under the ...