I am examining how a policy change impacted return on assets (ROA) using a difference-in-difference test. To address problems with serial correlation, we cluster standard errors at the firm level (since we have variation at firm-year level). We are not thinking of adding a robustness test where we bootstrap standard errors. Does it make sense to both cluster standard errors and bootstrap them, or should I just do bootstrapping and ignore clustering standard errors at the firm level in this robustness test. Furthermore, is this robustness test value adding?

Many thanks!


What you would want here is called a "block bootstrap." It is a type of bootstrapping that is specifically designed for situations in which clustering is an issue. There are various flavors of it. The basic gist of it though involves randomly drawing with replacement from your clusters (in your case firms) rather than from your individual observations. The idea here is that since the observations within each firm are highly correlated (and in particular, in the context of the DiD, essentially they each represent one observation of a single trajectory over the study period), you treat each firm as a single data point, rather than each observation within each firm. If you just did a standard bootstrap, the results would be badly misleading to your audience, so you'd be better off not doing that. But, doing a correctly formulated bootstrap is always a good robustness check for the reasonableness of your standard errors. If the results of the bootstrap and your errors that you calculate with a closed form expression are far off, it is a good sign that your model might be poorly specified (not to mention that the non-bootstrapped SEs are likely wrong in such a case).

For some very basic references with software examples, see here and here.

For a more in-depth and informative treatment, the article "Bootstrap-based Improvements For Inference With Clustered Errors" by A. Colin Cameron, Jonah B. Gelbach, and Douglas L. Miller is a good place to start. (see here).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.