9
$\begingroup$

Which setup is correct for a difference in difference regression model using

$Y_{ist} = \alpha +\gamma_s*T + \lambda d_t + \delta*(T*d_t)+ \epsilon_{ist}$

where T is a dummy which is equal to 1 if the observation is from the treatment group and d is a dummy which is equal to 1 in the time period after the treatment occurred

1) Random samples from each group and time (i.e. 4 random samples)

or

2) Panel data where the same units are tracked over both time periods?

Does it matter and if not, can OLS be used with either case?

$\endgroup$
  • 1
    $\begingroup$ I haven't seen (1) done - the analysis always seems = (2). Not sure why you would do (1). But I haven't seen many DID studies. $\endgroup$ – charles Dec 21 '13 at 4:28
  • 1
    $\begingroup$ Examples of 1 are shown in Wooldridge Introductory Econometrics section 13.2 $\endgroup$ – B_Miner Dec 21 '13 at 14:18
18
$\begingroup$

A key assumption of difference-in-differences (DID) is that both groups have a common trend in the outcome variable before the treatment. This is important in order to make the argument that the change for the treated group is because of the treatment and not because the two groups were already different from each other to begin with.

If you sample different people before and after the treatment this will weaken the argument unless your samples from the treatment and control groups are actually random and large. So it well might happen that somebody is going to ask you: "How can you make sure that the effect is due to the treatment and not just because you sampled different people?" - and that will be difficult to answer. This question you can avoid by using panel data because there you track the same statistical units over time and generally this is the more solid approach.

To answer your last question: yes the data matters but you can surely use OLS to estimate your equation above. An important thing which in the past was often overlooked is the correct estimation of the standard errors. If you don't correct them, serial correlation will underestimate them by a good amount and you will find significant effects even though you probably shouldn't. As a reference and suggestions for how to deal with this problem see Bertrand et al. (2004) "How Much Should We Trust Differences-In-Differences Estimates?".

As a last thing, if you have aggregate data (e.g. at the state level) or if you can easily aggregate yours and if you want to use a more recent econometric method than DID, you might want to have a look at Abadie et al. (2010) "Synthetic Control Methods for Comparative Case Studies". The synthetic control method is increasingly used in nowadays research and there exist well documented routines for R and Stata. Maybe this is something interesting for you as well.

$\endgroup$
  • $\begingroup$ This is great Andy! Can I summarize by saying that both data setups are acceptable but that panel data is the easier to make an argument about the assumptions? That both can be fit by OLS but that the standard errors of (especially the panel data setup I presume) are questionable due to possible serial correlation. Would a panel setup with Newey West SE be a good solution? $\endgroup$ – B_Miner Dec 21 '13 at 13:58
  • 6
    $\begingroup$ Yes, for the first data type you need more and strong assumptions. For the standard errors, the Newey West correction should work. Actually it's analogous to one of the correction methods proposed by Bertrand et al. (they use clustered standard errors). A more recent method uses the bootstrap which works pretty well (see rbnz.govt.nz/research_and_publications/seminars_and_workshops/…). Hope this helps! $\endgroup$ – Andy Dec 21 '13 at 14:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.