1
$\begingroup$

I have a panel structure with with three periods: pre, during, and post treatment. There are two ways running the DID approach:

1- The first one is to run one DID regression as the answer show in the following link.

Difference in Difference with multiple period (pre, during, and post treatment)

2- The second one is to run two DID regressions separately: comparing pre with during and comparing during with post period.

I know the results are a little bit different. Generally speaking, which method is more rigorous? Could someone help provide a list of classical papers (in literature) discussing advantages and disadvantages about these different ways? Thanks.

$\endgroup$
1
$\begingroup$

It is difficult to determine which is more rigorous, methodologically speaking. If you have multiple observations post-treatment, you could interact a treatment dummy with with all post-treatment periods (i.e., weeks/months/years). Here is the canonical DD setup,

$$ y_{st} = \gamma T_{s} + \lambda Post_{t} + \delta(Treat_{s} \times Post_{t}) + \epsilon_{st}, $$

where we observe a policy in effect in state $s$ during years $t$. Let's say a policy took effect at the start of 2016 and has remained in effect since its inception. The $Post_{t}$ variable should be coded 1 for each post-treatment period ($t$ = 2016 onward), irrespective of group status; you could also call this the "during" variable.

Some treatments/exposures remain in effect for the study period, while others come to a conclusion. Now let's tweak our hypothetical example a little. Say the policy ends at the conclusion of 2017, and you have observations for all states and years beyond conclusion of the intervention. You could proceed by excluding those years and focusing only on the post-treatment years (i.e., during period), or you could include a set of period dummies for all years (i.e., the "during" and "after" phase). Should you do this, you want to interact separate post-treatment indicators with treatment.

In the post you reference, Dimitriy's specification illustrates this very well. Deconstructing $Post_{t}$ into separate indicators for all periods after the policy takes effect results in the following

$$ y_{st} = \gamma T_{s} + \lambda_{1} (T_{s}*\mathbf{I}_{t = 2016}) + \lambda_{2} (T_{s}*\mathbf{I}_{t = 2017}) + \lambda_{3} (T_{s}*\mathbf{I}_{t = 2018}) + \lambda_{4} (T_{s}*\mathbf{I}_{t = 2019}) + \epsilon_{st}. $$

Again, treatment 'turns on' for two years (i.e., 2016 and 2017), then 'turns off' for the remainder of the study period. The interaction of treatment, $T_{s}$, with time dummies for 2018 and 2019 shows how the effect of treatment persists beyond conclusion of the intervention. We might see effects wane over time as the exposure is removed. Another way to approach this is to "lag" your treatment variable.

Some papers demarcate time into a Before, During, and After phase, with During serving as the $Post_{t}$ variable in the canonical DD setup. Please review the following dissertation paper on the web (Dissertation pp. 112-114) where this has been done.

Post-treatment is a bit of a misnomer, as one might think this indexes periods beyond some exposure period. But this is not the case in most applied work. The variable $Post_{t}$ typically indexes the full duration of some policy or intervention. I will let someone else jump in if they've seen differences in notation.

A paper by Ryan and colleagues (2015) recommend including a vector of pre-intervention dummies in the classical DD model (Paper, see equation 4 on page 1220). The paper is gated.

In sum, there is an advantage to modeling treatment in this way. The effect of treatment may not be constant over time.

$\endgroup$
  • $\begingroup$ The other advantage of single equation model is that it makes testing joint hypotheses easier. For example, you might want to test the null hypothesis that the effect in all post periods are jointly zero. You might want to test that the effects at the end is greater than the effect at the beginning. This is much harder if the the estimates come from different specifications. $\endgroup$ – Dimitriy V. Masterov Aug 29 at 21:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.