Resampling under the null versus the alternative hypothesis

Question

I'm looking at a community of organisms using simultaneous GLMs via the mvabund package. The manyglm function from this package fits a GLM for every species, each with a common set of predictors. The sum of likelihood ratios of each model (each species) is the test statistic, and resampling of observations is done to compute the significance.

Once the manyglm function is run, you can analyze the models with anova.manyglm or summary.manyglm. The difference between these two is that anova.manyglm resamples under the null hypothesis while summary.manyglm` resamples under the alternative hypothesis.

In this package, the resampling scheme is of residuals from the model. I'm making my way through "Bootstrapping Methods and their Applications" (Davison and Hinkley, 1997), but even after reading the chapters on boostrapping for a GLM, I'm still having trouble with the distinction.

So my question is what exactly is the difference between resampling under the null versus the alternative hypothesis?

Answer 1

Resampling under the null is used to get null distributions, which are, in turn, used for permutation tests of your model. Thus these are used to perform permutation (i.e. randomization) tests. It shows you what your analyses would look like if there are really no relationships in the data.

Resampling under the alternative is bootstrapping, probably to get confidence intervals. This is to show you what you might get if you replicate the study (assuming that there really IS a relationship among your variables).