13
$\begingroup$

If in standard OLS regressions two assumptions are violated (normal distribution of errors, homoscedasticity), is bootstrapping standard errors and confidence intervals an appropriate alternative to arrive at meaningful results with respect to the significance of regressor coefficients?

Do significance tests with bootstrapped standard errors and confidence intervals still "work" with heteroscedasticity?

If yes, what would be applicable confidence intervals that can be used in this scenario (percentile, BC, BCA)?

Finally, if bootstrapping is appropriate in this scenario, what would be the relevant literature that needs to be read and cited to arrive at this conclusion? Any hint would be greatly appreciated!

$\endgroup$
  • 1
    $\begingroup$ If there is such violation, I dont think bootstrapping cures it. Instead why not try to transform (log) the data to get closer to normality and use a robust standard error such as from the sandwich package in R? $\endgroup$ – B_Miner Apr 22 '13 at 15:14
  • $\begingroup$ The bootstrap works fine if you adapt the resampling scheme to the situation you're in. $\endgroup$ – Glen_b -Reinstate Monica Apr 23 '13 at 0:44
20
$\begingroup$

There are at least three (may be more) approaches to perform the bootstrap for linear regression with independent, but not identically distributed data. (If you have other violations of the "standard" assumptions, e.g., due to autocorrelations with time series data, or clustering due to sampling design, things get even more complicated).

  1. You can resample observation as a whole, i.e., take a sample with replacement of $(y_j^*, {\bf x}_j^*)$ from the original data $\{ (y_i, {\bf x}_i) \}$. This will be asymptotically equivalent to performing the Huber-White heteroskedasticity correction.
  2. You can fit your model, obtain the residuals $e_i = y_i - {\bf x}_i ' \hat\beta$, and resample independently ${\bf x}_j^*$ and $e_j^*$ with replacement from their respective empirical distributions, but this breaks down the heteroskedasticity patterns, if there are any, so I doubt this bootstrap is consistent.
  3. You can perform wild bootstrap in which you resample the sign of the residual, which controls for the conditional second moment (and, with some extra tweaks, for the conditional third moment, too). This would be the procedure I would recommend (provided that you can understand it and defend it to others when asked, "What did you do to control for heteroskedasticity? How do you know that it works?").

The ultimate reference is Wu (1986), but Annals are not exactly the picture book reading.

UPDATES based on the OP's follow-up questions asked in the comments:

The number of replicates seemed large to me; the only good discussion of this bootstrap parameter that I am aware of is in Efron & Tibshirani's Intro to Bootstrap book.

I believe that generally similar corrections for the lack of distributional assumptions can be obtained with Huber/White standard errors. Cameron & Triverdi's textbook discuss equivalence of the pairs bootstrap and White's heteroskedasticity correction. The equivalence follows from the general robustness theory for $M$-estimates: both corrections are aimed at correcting the distributional assumptions, whatever they may be, with the minimal assumption of finite second moments of residuals, and independence between observations. See also Hausman and Palmer (2012) on more specific comparisons in finite samples (a version of this paper is available on one of the authors' websites) on comparison between the bootstrap and heteroskedasticity corrections.

$\endgroup$
  • $\begingroup$ Thanks a lot for your help! Please allow me one followup question: The only assumptions that I violate are the normal distribution of errors and the homoscedasticity assumptions. Also, I am only interested in seeing whether my regression coefficients are sig. in the expected direction or no. The magnitude of the effect is not important. I think what I have done so far is your option 1. I bootstrapped standard errors and generated in addition bootstrapped confidence intervals. I did that using Stata: vce(bootstrap, reps(2500) bca), estat bootstrap. Does that cure my assumption violations? $\endgroup$ – David Apr 22 '13 at 20:05
  • $\begingroup$ I don't do diagnostics of the data based only on your syntax, and nobody will. What is the size of your data set? reps(2500) is probably an overkill, at least for the standard errors; I think reps(500) is OK for most practical purposes. Efron & Tibshirani's intro bootstrap book has a section on the number of replicates. They have a whole chapter on regression, too, so that may be another good reference for you to look at. $\endgroup$ – StasK Apr 22 '13 at 21:45
  • $\begingroup$ Thank you for your quick answer. The dataset is ~250. The questions on the number of replications aside (thank you for the link!), would you agree that bootstrapped standard errors (by way of resampling observations as a whole) and / or bootstrapped confidence intervals (e.g., percentile or bias corrected) would be an appropriate way to determine the significance (or lack thereof) of a regression coefficient given the violation of homoscedasticity and normal distribution of errors assumption? Thanks a lot for your input! $\endgroup$ – David Apr 22 '13 at 22:07
  • $\begingroup$ Yes, I would say that's better. If you use Stata though you could get a very similar answer using robust option of your regression. est store both results and est tab, se them to compare side by side. $\endgroup$ – StasK Apr 23 '13 at 0:26
  • $\begingroup$ Thank you StasK. I have also seen the following comment that you made somewhere else on this site: "Simple bootstrap with resampling ⇔ White's heteroskedasticity robust estimator". In the context of my questions as outlined above: Are there published journal articles that make this point? $\endgroup$ – David Apr 23 '13 at 23:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.