1
$\begingroup$

Is it okay to apply a generalized difference-in-differences (diff-in-diff) framework when the reference group has been treated from the beginning?

In my scenario, the study spans 4 time periods: $t = {1,2,3,4}$. Units in group 1 receive treatment throughout all 4 periods, whereas units in group 2 remain untreated in the first 3 periods but receive treatment at $t=4$.

I ran a generalized diff-in-diff framework, but I'm not sure if it makes sense:

$$ Y_{it} = \beta Treatment_{it} + Unit_{i} + Time_{t} + \epsilon_{it} $$

where $Treatment_{it}$ equals 1 for all units in group 1 at $t = {1,2,3,4}$ and for all units in group 2 at $t = 4$, and zero otherwise. $Unit_i$ and $Time_t$ are unit and time fixed effects.

Most online guides base their examples on a control group that is never exposed to treatment. However, in my case, the "control" group has been treated. So I'm unsure if I'm violating any assumptions or making any mistakes here. Any advice on this would be greatly appreciated.

$\endgroup$
2
  • $\begingroup$ Welcome. If the "comparison" group is treated all the time, then all units are either in a constant state of treatment or treated at time period 4? Is this correct? $\endgroup$ Commented Apr 26 at 15:06
  • $\begingroup$ If I interpret your question correctly, my answer is yes. All units in group 1 are in a constant state of treatment (i.e., treated at time periods 1,2,3,4), while all units in group 2 are only treated at time period 4. $\endgroup$ Commented Apr 26 at 15:11

1 Answer 1

2
$\begingroup$

Is it okay to apply a generalized diff-in-diff framework when the reference group has been treated from the beginning?

Not really.

I would advise you against incorporating units that were treated in all time periods. I suspect you're trying to exploit some treatment (event) of interest, whereby some units experience that treatment (event) by time period 4 and others do not. But note that the "always treated" are, by definition, always in a constant state of exposure to a stimulus. And even if they were untreated in the past, you do not have enough pre-event data to assess for any changes in their treatment history. We cannot approximate a counterfactual, which is what would have happened to the treatment group had no treatment ever been implemented, because our only observed "factual" state of the world is one where all units experienced the treatment. In short, you're missing that pre- versus post-treatment difference or "change" for a group of units that were never exposed to the treatment in periods 1–4.

Is your equation still estimable? Yes.

The generalized difference-in-differences framework will return a point estimate for $\beta$, regardless of how you code the control group. In other words, whether it's 'always 0' or 'always 1' doesn't matter with respect to identification, though 'always 1' is the proper coding scheme. In short, $Treatment_{it}$ should reflect the true state of the world. If a unit is "always treated" then it's equal to 1 in all time periods.

However, in my case, the "control" group has been treated. So I'm unsure if I'm violating any assumptions or making any mistakes here.

You cannot demonstrate certain assumptions due to insufficient data requirements. For example, difference-in-differences assumes parallel outcome trajectories between the treatment group and the control group prior to treatment exposure. In your setting, however, you do not observe a subset of units that never experienced treatment. Instead, what you have is a subset of units in a perpetually treated state. And even though these units were treated during your observation window, they weren't observed long enough in the past to establish whether any pre-exposure data were available. Note that difference-in-differences is comparing the before-and-after outcome variation exhibited among the treatment group with the before-and-after outcome variation exhibited among the control group. Not only is it nigh impossible to demonstrate parallel pre-treatment paths, but the trends among the "always treated" may bias the effects of units treated later.

So do you have to abandon this methodology entirely? Not exactly.

My suggestion would be to reverse-engineer the process. Let the "always treated" group constitute your control group. To be precise, we force the units exposed to the treatment (event) in all time periods to equal 0. On the other hand, the units that change their treatment status later constitute the treatment group. But note that the "true" state of the world is one where all units were treated by time period 4. Thus, the real action for identification of your treatment effect is in periods 1–3. You can allow $Treatment_{it}$ to equal 1 for treated units and only in periods 1–3, 0 otherwise. In fact, you can code the "time" component as you did earlier and allow it to 'switch on' in time period 4; this won't affect the point estimate, only the sign. If estimated both ways, $|\widehat{\beta}|$ should be identical.

By allowing a condition where the "always treated" represent the controls and the treated are those units that switch into the treatment condition later, then the parallel paths assumption must hold after the change. One may also refer to this identification condition as requiring future parallel treated paths. I have discussed different applications of difference-in-differences in reverse (or DDR) in my responses here and here.

As a final word, demonstrating a future trend equivalence may be difficult (if not impossible) even if this exposure process is reversible. Now your data requires at least two additional time observations beyond period 4. For example, observing units for at least 1–6 periods should suffice in my humble opinion.

If studies require at least 3 time periods to establish past parallel "untreated" paths, then I would argue the same number of time periods should be required to establish future parallel "treated" paths.

$\endgroup$
1
  • $\begingroup$ Thanks! This is really helpful! $\endgroup$ Commented Apr 29 at 12:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.