I was trying to solve a question in one of the exercises in my class regarding bootstrap sampling but I am stuck with 0-idea how to move forward. The question is about bootstrapping and calculating the bootstrap and the out-of bootstrap estimates of the generalization error. There are 4 samples which are shown below;

Original Data |Z* | Bootstrap 1 |Bootstrap 2 |Bootstrap 3 | Bootstrap 4
Z₁---------------- Z₁*= ---Z₄--------------- Z₅ ---------------Z₃------------ Z₃
Z₂---------------- Z₂*= ---Z₁--------------- Z₅ ---------------Z₁------------ Z₅
Z₃---------------- Z₃*= ---Z₄--------------- Z₂ ---------------Z₃------------ Z₂
Z₄---------------- Z₄*= ---Z₂--------------- Z₁ ---------------Z₄------------ Z₂
Z₅---------------- Z₅*= ---Z₁--------------- Z₃ ---------------Z₂------------ Z₃

Questions can be found here: http://imgur.com/a/sNEZv


In general, the formula allows one to compute an estimate of the generalization error by averaging the error over the 4 Bootstrap data sets.

(i) The estimator $\hat{m}^{*(j)}$ is simply computed based on the $j^{th}$ Bootstrap set. The estimator is identical for both, the bootstrap and the out-of-bootstrap methods.


  • In the standard bootstrap method, we average over all bootstrap samples, i.e. $\alpha_{ij} = \frac{1}{n} = \frac{1}{5}$.
  • For the out-of-bootstrap case, the inner sum is only over those elements that were not used for training, i.e. the elements that are not contained in $j^{th}$ bootstrap data set. Therefore, $\alpha_{ij} = 0$ if $Z_i \in \text{Bootstrap } j$ otherwise $\alpha_{ij}$ is equal $\frac{1}{\# \{\text{distinct elements in Bootstrap j\}}}$.
  • $\begingroup$ We already have our bootstrap samples. For Standard bootstrap case shouldn't the calculation for $\alpha_{ij}$ also depend on the drawings bootstrap sample? $\endgroup$ – NoResultOnTheWeb Aug 13 '17 at 13:52
  • $\begingroup$ No, only the estimator depends on the bootstrap samples. The bootstrap estimate for the generalization error is computed based on all training data. $\endgroup$ – aratar Aug 14 '17 at 11:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.