2
$\begingroup$

My lecture note says

Since we add residuals to the fitted model, this residual bootstrap only works for a homoscedastic regression model, where the error distribution does not depend on the predictors. This means it cannot be used for GLM.

My question is what is "error distribution" in the GLM? And why "error distribution depends on the predictors" so that "residual bootstrap does not work for GLM"?

$\endgroup$

1 Answer 1

4
$\begingroup$

It won't work with raw residuals for several reasons, but arguably can work - at least for some cases - with other choices. E.g pearson residuals can take care of the heteroskedasticity issue.

Other choices can reflect other aspects as well E.g. gamma regression often seems to work fairly well with anscombe residuals or deviance residuals. For discrete data you may need to either round to integers or go to a quasi-glm.

There's also the possibility of simple row-resampling, though I would only consider it with a lot of data.

Then there's parametric bootstrapping.

Check out the book by Davison and Hinkley for examples of bootstrapping glms

There isn't really an error distribution in a glm in the sense of the thing a residual approximates - only the Gaussian has additive constant variance error, arguably the gamma has a multiplicative error, but most glms the residuals change shape as well as variance as the mean changes. I'd strongly advise not trying to conceive of glms in terms of error in general, but instead in terms of conditional distributions.

$\endgroup$
4
  • $\begingroup$ What is row resampling, bootstrapping the indices of the XY data frame? This then treats the predictors as a random variable. Is that why you’d want a large data set? $\endgroup$
    – Dave
    Sep 7, 2020 at 4:12
  • $\begingroup$ Yes sampling the indices. You still condition on the observed x's in the resample, so they're not random within the fit, only across resamples. This doesn't work for a designed experiment but makes sense for observational data. $\endgroup$
    – Glen_b
    Sep 7, 2020 at 4:25
  • $\begingroup$ Why "it won't work for raw residuals"? $\endgroup$
    – WCMC
    Sep 7, 2020 at 4:43
  • 1
    $\begingroup$ The first sentence of your quote gives one reason, but just consider a gamma or a Poisson regression - especially which includes small values - or a logistic regression and actually try it. I'm sure several issues will become apparent quite quickly. What happens for example, when you resample a negative residual that's larger than a fitted mean? What's your bootstrapped data value? How does a GLM fitting routine respond in any of those cases to such a data value? $\endgroup$
    – Glen_b
    Sep 7, 2020 at 17:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.