Robust Coefficients For Differences in Differences

Question

I have a panel data set which I am looking to analyze for relationships/causality using the OLS differences-in-differences method. The panel data includes multiple observations over time for various groups that are most likely correlated to each other. Serial autocorrelation also exists within the data, given the nature of the time series component. My goal is to make my analysis as robust as possible. I’ve researched ways to prevent a biased estimation in the coefficients as much as possible due to these effects, but I am struggling with finding and applying a singular approach.

Example: Lagging the time series component and introducing the lag’d value into the model will minimize the residual autocorrelation attributed to the time series in the model. But won’t account for the between-group relationships. Using clustered standard errors makes the coefficients more robust, but doesn’t seem to deal with the time series autocorrelation component (at least as far as I can tell). Can I obtain the best (i.e. most robust and accurate) result using both methods? Or would doing so further introduce random noise/bias in the model that I’m just missing? Would just one be more appropriate?

Bonus: Is there a Bayesian approach to solve this problem?

Answer 1

First we should probably clear out the distinction between a bias in the coefficients and a bias of the standard errors.

In order to obtain an unbiased estimate of the treatment effect in a difference in differences setting you need the parallel trends assumption to hold. See here for a detailed explanation. If this assumption holds, treatment selection is not endogenous, and you don't have any time varying factors which affect the outcome deferentially across treatment groups at the same time your treatment kicks in (i.e. no other confounding factors), then you will get an unbiased coefficient.

The adjustment to standard errors to account for serial correlation and heteroscedasticity is an entirely different matter. This is not about correct estimation of the treatment effect coefficient but about the correct estimation of its standard errors. For appropriate inference in difference in difference settings you can have a look at Bertrand et al. (2004) "How much should we trust difference-in-differences estimates?" (link, ungated working paper version here).

One approach they suggest in order to correct for both autocorrelation and heteroscedasticity is to cluster standard errors on the panel unit id. This is easily implemented in most statistical packages. If your unit identifier is id then in Stata you would just use the variance-covariance estimation option cluster(id) at the end of the regression command. To me this seems to be the "singular approach" that you were looking for. One word of caution regarding lagged dependent variables: since difference in differences is a type of fixed effects regression adding a lagged dependent variable requires very strong assumptions in order to yield unbiased estimates (you can find a discussion of the issue in any book on panel data econometrics such as Angrist and Pischke (2009) or Wooldridge (2011)).