385 views

### How to compare 2 series of ROC curves with AUC? [duplicate]

I am testing two predictive models, Each of them in 10 experiments. I therefore have 10 ROC curves (with AUC) for first experiment and 10 ROC curves for second experiment. How to compare these ...
129 views

### How to test if there is any significance difference between the areas under under two curves [duplicate]

I am trying to test if there is any significant difference between the areas under two curves. x-axis: length of stay (los) y-axis: percentage curve1: control group curve2: intervene Any idea how ...
1 vote
123 views

### Diagnostic accuracy test done using mada package. How can I compare the AUC of SROC curves? [duplicate]

I am performing a meta-analysis to compare the diagnostic accuracy of different modalities on the same population. I have constructed SROC curves and have calculated the AUC values in each case. Is ...
1 vote
62 views

### Wanted: Statistical test to compare array with single value [duplicate]

I am comparing ROC AUC values for different feature selection methods. Some methods are deterministic and output the same AUC score every time and some methods are non-deterministic and output ...
18 views

### Can you check the significance of your AUC values? [duplicate]

I have computed some AUC values from the ROC curve based on logistic regressions. Firstly, I have divided my two datasets (D1, D2...
13k views

### What statistical tests to compare two AUCs from two models on the same dataset?

Let say I build two machine learning classifiers, A and B, on the same dataset. I obtain the ROC curves for both A and B, and the AUCs value. What statistical tests should I use to compare these ...
1 vote
5k views

### Confidence Interval - Binary classification [duplicate]

How do we calculate a confidence interval for a result in binary classifiers ? CI for regression problems makes sense since we have a variable estimated output that I can calculate its estimated mean ...
1 vote
9k views

### Calculate AUC using sensitivity and specificity values only

How to calculate AUC, if I have values of sensitivity and specificity for various threshold cutoffs? I have sensitivity and specificity values for 100 thresholds. ...
1 vote
850 views

### Is the AUC a robust metric to determine which network architecture to use?

I would like to figure out which hyperparameters/network architectures work best on a specific binary dataset. To compare them, I do k-fold cross-validation, use different MLP or ConvNet ...
1k views

### Accuracy and ROC for Logistic and Decision Tree

So I run a logistic regression and decision tree model using same data. The accuracy shows that the decision tree outperforms logistic slightly. However, my ROC curve shows that logistic is much ...
305 views

### Finding a statistically significant difference between two AI models?

I have created a machine learning model (let's call it Model B) which I hope is better at classifying disease-states than the currently used model (Model A). I have a dataset of 600 cases (roughly ...