Bootstrap ANCOVA/regression: what to do with residuals from different groups? I find myself wanting to solve a standard ANCOVA-style problem that has a factor with two levels and then the covariate(s). Rather than standard methods of doing inference on the group parameter, I want to try a bootstrap approach.
The way I have learned to bootstrap a regression is to bootstrap the residuals, not the original (multivariate) data. I do not have a fancy experimental design, but it is fair to call what I am doing a designed experiment rather than an observational study, so I do believe that my predictor variables are not random variables; they would be the same if I reran the experiment. However, I find myself in some funky territory. Residuals can come from either of the two groups.
What do I do with that? Do I let the residuals from the groups mix? Do I take my bootstrap samples from the two groups separately and then pool them to do the regression?
(I may be using a quantile regression approach, so the usual F-test and t-based confidence intervals are not quite right.)
 A: Despite its "nonparametric" flavor, you have to make some assumptions when you bootstrap. To get valid asymptotic results, you have to bootstrap in a way that mimics the true data-generating process. For example, standard bootstrapping is done by sampling independently, so you are making a strong assumption that the true data-generating process produces independent observations when you bootstrap.
In your case, the issue of whether to pool or sample within-group residuals separately boils down to an assumption of constant distributions of residuals (which implies that the distributions of the actual $Y$ data are rigidly location-shifted) across the groups and covariate values. This is a pretty strong assumption - even stronger than simple homoscedasticity.
You can indeed partly avoid this assumption by sampling residuals within groups, but realize that you are still making the quite strong assumption that the distributions of $Y$ are rigidly location-shifted across the values of the covariate within a group.  Also, separately bootstrapping has the distinct disadvantage of reducing the sample sizes, so your within-group estimates of the distributions are less reliable. This can make the performance of the bootstrap worse, even under heteroscedasticity (cf, variance/bias trade-off).
The vector resampling approach, where you vector resample the $(Y, T, C)$ intact avoids these assumptions (except independence), but as you mention, it does not mimic the data- generating process in that your treatment indicator ($T$) is truly fixed, but this approach makes it random.
I would recommend the pooled sampling of residuals as a first option, realizing its pitfalls of course.
Depending on what you are doing with your analysis, there is a decent chance that it will make little difference as compared to the classic approach.
See:
On using the bootstrap for multiple comparisons, J Biopharm Stat 2011 Nov;21(6):1187-205.
