I would like to construct CIs for $\beta$ in the linear model

$Y = X\beta + \epsilon$

I observe $\{X', Y'\}$ which is $\{X,Y\}$ contaminated with values missing at random. $\epsilon$ is not Gaussian and not homoscedastic.

I propose to construct CIs for $\beta$ by:

  1. Generating several imputed data sets $\{X^i, Y^i\}_{i \in {1, \ldots, N}}$ using, say, multiple imputation by chained equations;
  2. Constructing bootstrap replicates of $\hat\beta$ by randomly selecting an imputed data set then, based on this dataset, generating a bootstrap of the cases replicate of $\hat\beta$.(A new dataset is randomly selected for each individual replication).
  3. Using a standard method to construct a CI from the bootstrap replicates, say, Efron's bias corrected accelerated CIs.

Is this the best way to go about it?


Shao and Sitter 1996 demonstrate that the right approach is:

  1. Take a bootstrap sample, respecting the dependencies in the data (see below);
  2. Run one imputation on this sample, estimating the imputation model and producing one model + noise replicate;
  3. Run a complete case analysis on this;
  4. Repeat 1-3 $B$ times;
  5. Combine using the bootstrap rules (not the Rubin rules).

$B$ must be bootstrap-large, not the Rubin-large... 5 hundred rather than 5. The biggest issue that comes up with complex survey data which are in the focus of Shao & Sitter's paper is that there are non-trivial dependencies and indepedencies present in complex survey data. By design, observations between strata are independent, and imputation that borrow strength across the whole data set violate that independence. By design, observations within the same PSU are correlated. Both of these effects need to be addressed by the bootstrap scheme. For complex surveys, this needs to be the complex survey bootstrap. For time-series, this needs to be the block bootstrap.

The process proposed by orizon (as clarified by Stef) may be right, and I have been rolling it in my head for some while in the past couple of years, but never had the chance to really review it for statistical soundness.

  • $\begingroup$ That is a good point and seems to me superior to the strategy I outlined (as improved/clarified by Stef). Ultimately I followed Efron (1994), which is very close to your reiteration of Shao & Sitter (ie imputation comes after bootstrap sampling not before). $\endgroup$ – orizon Aug 8 '13 at 22:56

Steps 2 and 3 ignore the fact that some of the data have been imputed. Hence the bootstrap estimate of the distribution of $\hat\beta$ will be too narrow.

Rubin's pooling rules combine the within and between imputation uncertainty. Though this procedure assumes that $\hat\beta$ is normally distributed around the population value $\beta$, it is actually quite robust against violations of normality.

  • 1
    $\begingroup$ I think that steps 2 and 3 do account for the variability attributable to imputation because each bootstrap replicate is based on a different imputation (really some will overlap but if N is chosen sufficiently large this is not much of a problem). I have clarified step 2. While I am pleased that pooling is robust, I don't think it will work in my case because the singly imputed bootstrap CIs are asymmetric. $\endgroup$ – orizon Apr 15 '13 at 12:05
  • 2
    $\begingroup$ Thanks for the clarification. In the previous version I missed the step where you redraw the multiply-imputed data for every replication. You will probably need large m to incorporate adequate between-imputation variability. If I understand correctly you will have the following procedure: 1) generate one imputed data set, 2) take a bootstrap sample from that 3) estimate beta 4) repeat 1-3 1000 times, 5) summarize the distribution of 1000 beta's. I think something like this might work, but you really ought to do some simulation to confirm it. $\endgroup$ – Stef van Buuren Apr 15 '13 at 19:05
  • $\begingroup$ Thank you, I will run some simulations and see if it works well or not. $\endgroup$ – orizon Apr 16 '13 at 0:11
  • $\begingroup$ Stef, see my answer for an alternative treatment. $\endgroup$ – StasK Aug 8 '13 at 13:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.