Linked Questions
0
votes
0
answers
387
views
How to compare 2 series of ROC curves with AUC? [duplicate]
I am testing two predictive models,
Each of them in 10 experiments.
I therefore have 10 ROC curves (with AUC) for first experiment and 10 ROC curves for second experiment.
How to compare these ...
- 693
2
votes
0
answers
132
views
How to test if there is any significance difference between the areas under under two curves [duplicate]
I am trying to test if there is any significant difference between the areas under two curves.
x-axis: length of stay (los)
y-axis: percentage
curve1: control group
curve2: intervene
Any idea how ...
- 21
1
vote
0
answers
123
views
Diagnostic accuracy test done using mada package. How can I compare the AUC of SROC curves? [duplicate]
I am performing a meta-analysis to compare the diagnostic accuracy of different modalities on the same population. I have constructed SROC curves and have calculated the AUC values in each case. Is ...
- 11
1
vote
0
answers
63
views
Wanted: Statistical test to compare array with single value [duplicate]
I am comparing ROC AUC values for different feature selection methods. Some methods are deterministic and output the same AUC score every time and some methods are non-deterministic and output ...
- 121
0
votes
0
answers
18
views
Can you check the significance of your AUC values? [duplicate]
I have computed some AUC values from the ROC curve based on logistic regressions. Firstly, I have divided my two datasets (D1, D2...
- 528
15
votes
2
answers
16k
views
What statistical tests to compare two AUCs from two models on the same dataset?
Let say I build two machine learning classifiers, A and B, on the same dataset.
I obtain the ROC curves for both A and B, and the AUCs value.
What statistical tests should I use to compare these ...
- 13k
1
vote
2
answers
6k
views
Confidence Interval - Binary classification [duplicate]
How do we calculate a confidence interval for a result in binary classifiers ?
CI for regression problems makes sense since we have a variable estimated output that I can calculate its estimated mean ...
- 11
1
vote
2
answers
11k
views
Calculate AUC using sensitivity and specificity values only
How to calculate AUC, if I have values of sensitivity and specificity for various threshold cutoffs?
I have sensitivity and specificity values for 100 thresholds.
...
1
vote
1
answer
916
views
Is the AUC a robust metric to determine which network architecture to use?
I would like to figure out which hyperparameters/network architectures work best on a specific binary dataset. To compare them, I do k-fold cross-validation, use different MLP or ConvNet ...
- 133
2
votes
0
answers
1k
views
Accuracy and ROC for Logistic and Decision Tree
So I run a logistic regression and decision tree model using same data. The accuracy shows that the decision tree outperforms logistic slightly. However, my ROC curve shows that logistic is much ...
- 31
2
votes
2
answers
320
views
Finding a statistically significant difference between two AI models?
I have created a machine learning model (let's call it Model B) which I hope is better at classifying disease-states than the currently used model (Model A).
I have a dataset of 600 cases (roughly ...
- 31
4
votes
0
answers
510
views
Compare and quantify relative improvement in ROC AUC scores?
What is an appropriate method for comparing relative improvement in model performance across different problems?
For example, say I have three different datasets/problems a, b, c, and two models for ...
- 649
0
votes
0
answers
643
views
Comparing AUCs for two ROC curves
I have a data analysis puzzle involving ROC curves that I hoped you could help me with.
One of my research projects involves exploring how to use crowds to do idea filtering (i.e. to distinguish good ...
- 1