Linked Questions

0 votes
0 answers

How to compare 2 series of ROC curves with AUC? [duplicate]

I am testing two predictive models, Each of them in 10 experiments. I therefore have 10 ROC curves (with AUC) for first experiment and 10 ROC curves for second experiment. How to compare these ...
suprvisr's user avatar
  • 673
2 votes
0 answers

How to test if there is any significance difference between the areas under under two curves [duplicate]

I am trying to test if there is any significant difference between the areas under two curves. x-axis: length of stay (los) y-axis: percentage curve1: control group curve2: intervene Any idea how ...
Erika Sama's user avatar
1 vote
0 answers

Diagnostic accuracy test done using mada package. How can I compare the AUC of SROC curves? [duplicate]

I am performing a meta-analysis to compare the diagnostic accuracy of different modalities on the same population. I have constructed SROC curves and have calculated the AUC values in each case. Is ...
user86013's user avatar
1 vote
0 answers

Wanted: Statistical test to compare array with single value [duplicate]

I am comparing ROC AUC values for different feature selection methods. Some methods are deterministic and output the same AUC score every time and some methods are non-deterministic and output ...
mcb's user avatar
  • 121
0 votes
0 answers

Can you check the significance of your AUC values? [duplicate]

I have computed some AUC values from the ROC curve based on logistic regressions. Firstly, I have divided my two datasets (D1, D2...
Thomas's user avatar
  • 468
13 votes
2 answers

What statistical tests to compare two AUCs from two models on the same dataset?

Let say I build two machine learning classifiers, A and B, on the same dataset. I obtain the ROC curves for both A and B, and the AUCs value. What statistical tests should I use to compare these ...
RockTheStar's user avatar
  • 12.5k
1 vote
2 answers

Confidence Interval - Binary classification [duplicate]

How do we calculate a confidence interval for a result in binary classifiers ? CI for regression problems makes sense since we have a variable estimated output that I can calculate its estimated mean ...
acer_7's user avatar
  • 11
1 vote
2 answers

Calculate AUC using sensitivity and specificity values only

How to calculate AUC, if I have values of sensitivity and specificity for various threshold cutoffs? I have sensitivity and specificity values for 100 thresholds. ...
Dhwani Dholakia's user avatar
1 vote
1 answer

Is the AUC a robust metric to determine which network architecture to use?

I would like to figure out which hyperparameters/network architectures work best on a specific binary dataset. To compare them, I do k-fold cross-validation, use different MLP or ConvNet ...
jerpint's user avatar
  • 133
2 votes
0 answers

Accuracy and ROC for Logistic and Decision Tree

So I run a logistic regression and decision tree model using same data. The accuracy shows that the decision tree outperforms logistic slightly. However, my ROC curve shows that logistic is much ...
YY_H's user avatar
  • 31
2 votes
2 answers

Finding a statistically significant difference between two AI models?

I have created a machine learning model (let's call it Model B) which I hope is better at classifying disease-states than the currently used model (Model A). I have a dataset of 600 cases (roughly ...
schmitzhugen's user avatar
4 votes
0 answers

Compare and quantify relative improvement in ROC AUC scores?

What is an appropriate method for comparing relative improvement in model performance across different problems? For example, say I have three different datasets/problems a, b, c, and two models for ...
Bryan Shalloway's user avatar
0 votes
0 answers

Comparing AUCs for two ROC curves

I have a data analysis puzzle involving ROC curves that I hoped you could help me with. One of my research projects involves exploring how to use crowds to do idea filtering (i.e. to distinguish good ...
Lukgaf's user avatar
  • 1