My question is related to bootstraping and extracting confidence interval for classification problems. Let's say I have 25 number of data points and 2 features and use Gradient Boosting Machine (GBM) to develop a binary classifier. Then I extract AUC to examine what's the accuracy of my model. When I do bootstraping by reshuffling the data points (basically changing order of data points), I could get different AUC and I do this work for 1000 times and then will find 95% confidence interval and p-value. But, in my opinion, it's something nonsense. Essentially, why my predictive model (i.e. GBM) should depend on order of data points that I use for training? I mean how do I know which order is correct cause my data points do not have any order. I can't wrap my head around this and I will be very grateful if someone could explain it to me.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.