In Ordinary bootstrap, people draw bootstrap samples from the original sample and fit models using bootstrap samples. Those models are further applied to the original sample to get performance values and the average is the estimate of the likely performance on the future data. Efron further proposed an enhanced bootstrap in which the value of optimism is obtained from the difference between the model on bootstrap model and original sample. This optimism is used to correct the bias of model on original sample to get an accurate estimate in the population level. (if feature selection is included, then the process building the model is validated)

The optimism mentioned above is calculated as the following: Fit a model based on a bootstrap sample and a performance index(named as A) is obtained at the same time. The same model is applied to the original sample to get a performance index called B. The estimate of the optimism would be the difference between B and A, B-A. According to the principle of the bootstrap, the bias-corrected performance would be B+(B-A) = 2B-A.

Bootstrap OOB is more often used in aggregated bagging or boosting in machine learning. Samples not picked up in bootstrap serves as test data to evaluate the performance of the model on the bootstrap sample.

My question I can understand the mechanism of bias-corrected bootstrap since "bootstrap resample to original sample is as the sample to the population", but I can't understand the rationale of bootstrap-OOB. Both ways are likely to be used to validate a model or a process, what's the difference between them?

  • $\begingroup$ This is a neat question, but it's a little hard to understand. Could you explicitly define "optimism"? And could you put paragraph breaks where you change topics? $\endgroup$ – eric_kernfeld Jan 15 at 21:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.