How does case-resampling bootstrap work for positive-value estimators?

Question

I've looked at some other questions on bootstrap significance testing:

• Non-parametric bootstrap p-values vs confidence intervals
• Computing p-value using bootstrap with R
• p-value vs. confidence interval obtained in Bootstrapping

...and mine is a bit different. I think my problem might be specific to cases where the parameters under the null are on the boundary of the parameter space.

Suppose you do a case-resampling bootstrap, where you resample observations with replacement, fit a model, and get a bootstrap distribution for the parameter you are trying to estimate.

Now suppose the parameter and your estimator are strictly positive. For example, REML estimates of variance of random effects in random/mixed effect models, or estimates of information gain in classification trees. I have found often that often the canonical null hypotheses in these cases is is that the parameter = 0.

Example: Random Effects Modeling

I'll use a random effects model as a specific example. A parametric bootstrap to test the hypothesis that the variance of the random effect is 0 might go as follows:

1. Fit the model with the random effect using maximum likelihood.
2. Do the same for the model without the random effect.
3. Calculate a likelihood test statistic
4. Repeat steps 1:3 using a response simulated with resampled errors. Do this n times, obtaining a bootstrap distribution of likelihood test statistics
5. Calculate the bootstrap p-value of the original test-statistic using the bootstrap distribution.

Now suppose you want to use a REML estimator. The model with and without the random effect are not comparable using the likelihood test. So why not try a case-resampling bootstrap? In a case-resampling bootstrap, one would obtain a bootstrap distribution of variance estimates for that random effect using data resampled with replacement. But then what?

If we were trying to do inference on a regression coefficient, one might conclude in favor of the null that a regression coefficient is 0 if the bootstrap confidence interval contains 0. But this is never the case for an estimator that is >= 0. How does one evaluate the significance of the estimate in this case?

Answer 1

The traditional bootstrap fails when the parameter is on the boundary of the parameter space, as in the mixed model example of estimating the variance of random effects. See http://www.jstor.org/stable/2999432. You'd have to do something smarter, which leads you to the parametric bootstrap where you'd have to effectively recreate your model under the null with no random effects -- see this post for additional discussion and additional references.