For my thesis I would like to compare different supervised machine learning algorithms based on their AUC score. My supervisor pointed out to also report the confidence intervals (CI) of the AUC scores. He told me that there is an article in which they calculate the CI based on the positive and negative samples in the dataset.
I think i have found the right article: Cortes, Corinna, and Mehryar Mohri. "Confidence intervals for the area under the ROC curve." Advances in neural information processing systems. 2005.
Although there are a lot of formula's in the article i am still not able to figure out how to 'simply' do it..
Could somebody help me out with this, or has a good example for it. Some numbers:
Training set: positive samples: 307 negative samples: 21156
Test set: positive samples: 77 negative samples: 21463
For me the ideal would be to calculate the CI without a bootstrapping method ad the data is on a computer which is not very accessible.
EDIT: SOLUTION I WAS LOOKING FOR, i think;): The answer i think i was looking for was in the article but through another article i sort of 'got it' the confidence interval can be calculated with the formula below.