I'm trying to bootstrap non-parametric prediction errors for a model I'm building. My understanding is that the procedure below should yield bootstrapped predictions and prediction errors, with the input matrix $X$, response vector $y$, an input matrix to be predicted $X_p$, and $N$ equal to the number of resamples (e.g. 10,000):

for $i$ in $N$:

  1. Resample the rows of $X$ with replacement to get $X_i$
  2. Retrain the model $m_i(X_i)$ and predict the responses: $\hat{y_i} = m_i(X_p)$

Then the bootstrap estimate and prediction intervals can be computed directly from that vector (e.g. mean and quantiles of the vector).

However, I've seen several sources that seem to suggest that I might need to resample from the $\epsilon_i = y - \hat{y_i}$ and rebuild the model again based on that. Can someone clarify the proper procedure here?


Sources: Slides 12-15 https://www.emse.fr/~roustant/Documents/Bootstrap_Conf_and_Pred_Intervals.pdf

Bootstrap prediction interval (Though this method seems somewhat more complicated and involves resampling the $\epsilon$s and rebuilding the model based on that.)

  • $\begingroup$ For linear regression and more generally there are two method given in my book or Efron and Tibshirani. They are called bootstrapping vectors and bootstrapping residuals. In the first edition of my book published in 1999 the advantages and disadvantages of each are described on pages 70-74. $\endgroup$ – Michael Chernick Jul 2 '18 at 1:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.