I am using LOOCV to calculate AUC and using the bootstrap BCa approach to calculate the confidence intervals. Now, I've read that the confidence intervals drop by root(N), is N the number of bootstraps or the number of samples in the data? What I am trying to do is to see how the number of patients in the data affects the AUC performance if I were to use the exact same model (only update the hyperparameters). For example, if I have 300 patients and I got an AUC of 0.9 +/- 0.05, how will that change if I now have 350 (to say a number)? - Assuming my model is very stable

  • 3
    $\begingroup$ $N$ is sample size, not number of bootstrap samples. $\endgroup$ – kjetil b halvorsen Jan 22 at 12:25
  • $\begingroup$ So will the confidence interval drop by root(N) no matter what approach I use (percentile, BC, BCa)? It seems to me that it is only true for the percentile approach $\endgroup$ – Luis Pinto Jan 22 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.