Linear Model Residual Bootstrapping

Question

I am having some issues how to figure out what the second iteration of a fixed-x resampling bootstrap would be. For instance if I have this data

Y   x1  x2
7   2   3
6   3   5
5   4   1
9   5   4

and I calculate the coefficients

        Coefficients
Intcp   3.137931034
x1      0.551724138
x2      0.517241379

Getting the following residuals:

1.206896552
-1.379310345
-0.862068966
1.034482759

On my first bootstrap iteration I sample the residuals and add those samples residuals to the Y from the previous model. I sampled -0.862068966, -1.379310345, -0.862068966, 1.206896552 in this case. Getting the following table:

Y           x1  x2
6.137931034 2   3
4.620689655 3   5
4.137931034 4   1
10.20689655 5   4

Which leads to the coefficients:

   
            Coefficients
Intercept   -0.096313912
x1          1.235434007
x2          0.63020214

and residuals of

1.206896552
-1.379310345
-0.862068966
1.034482759

For the next bootstrap, am I supposed to continue this chain and sample the last set of residuals and add them to the last set of Ys? Or do I use either the original Y values or original residuals in anyway?

Answer 1

In bootstrapping, you repeatedly sample with replacement from the original sample. You can simply add this to your loop:

wh <- sample(1:nrow(original.sample), replace = TRUE)
boot.sample <- original.sample[wh, ]
boot.lm <- lm(Y ~ x1 + x2, data = boot.sample)

For fixed-x, or residual resampling as you intend, you sample with replacement from the original residuals in each iteration.

residuals <- resid(model)
wh <- sample(1:length(residuals), replace = TRUE)
boot.Y  <- original.Y + residuals[wh]
boot.lm <- lm(boot.Y ~ x1 + x2)