1
$\begingroup$

Would a difference-in-differences analysis still tell me something important if the control group was treated later in time? Or, would I be better off only restricting my analysis to the time frame up to when the control group was treated with that same treatment the treatment group underwent?

$\endgroup$
2
  • $\begingroup$ Do all units eventually become treated? $\endgroup$ Commented May 2, 2022 at 6:40
  • 1
    $\begingroup$ Is the decision to treat based on outcomes? E.g. one particularly tricky case for cancer drugs, where a new/different treatment will usually be tried when disease progresses. That usually complicates analyses of survival time even in randomized controlled trials, so this only gets worse in observational studies. If the "treatment" is given in your application for reasons totally unrelated to previous or predicted outcomes, then your situation is a lot simpler. $\endgroup$
    – Björn
    Commented May 2, 2022 at 7:53

1 Answer 1

1
$\begingroup$

Would a difference-in-differences analysis still tell me something important if the control group was treated later in time?

Yes it would.

Your treatment effect is identified in terms of variation in treatment timing. If the "control group" is treated farther downstream, then I assume all units end up in the "treatment condition" eventually. Thus, you don't have a true baseline history of "non-adopter" units in your setting. Your effect is driven by the different timing impacts across units.

A note about treatment effect heterogeneity. You're making some "forbidden comparisons" in this setting. Before the first treatment, all other units "not yet treated" may serve as a counterfactual. This is type of comparison is okay. On the other hand, as time goes on, the "previously treated" also serve as a counterfactual for those treated much later. Even with a staggered exposure design, we still assume constant treatment effects. Bias may be introduced once the effects start to change over time.

Or, would I be better off only restricting my analysis to the time frame up to when the control group was treated with that same treatment the treatment group underwent?

This is often recommended.

However, I recommend comparing the "early-adopter" units with a group/cohort of "non-adopter" units, and a second comparison of "late-adopter" units with the "non-adopter" units. The "late-adopters" are just those units that you refer to as "treated later" in your panel. But notice how I am assuming you actually observe some units in the control condition in all time periods. If, however, all units eventually become exposed to the treatment, then this process is a bit clunky. Before each exposure, the cohort of units entering into the treatment condition is compared with any unit theretofore untreated. You may want to exclude the units with a previous history of treatment, such as those treated farther upstream. Assuming you have clearly defined waves of treatment, then each timing group is getting a clean comparison with a unit(s) that have yet to undergo a treatment. On the other hand, if units move in and out of treatment at widely different times, or even multiple times over time, then this approach becomes a bit unwieldy. My recommendation assumes treatment is permanent. Estimators do exist to handle treatment reversal, but this is outside the scope of your question. I would need to learn more about the treatment's variation over time to offer more guidance.

If you really only have the "early-adopter" and "late-adopter" cohorts, then I'm not sure why the comparison you're proposing is helpful. By the second wave of treatment, you're left with comparing the "late-adopter" units with the "early-adopter" units. In other words, you're limiting the analysis to units with a treatment history. If this is qualitatively the same treatment that the "early-adopter" units received upstream, then this type of analysis is meaningless. By truncating the series to the epoch where the "late-adopter" units first undergo a treatment, you've removed all variation in treatment timing. The counterfactual for the cohort treated "late" is the cohort treated early.

As a final word, we still assume parallel trends across the different timing groups. It's not uncommon in quasi-experimental evaluations to observe a previous exposure affect the trends downstream. In reality, the magnitude of the first shock may even make the "early-adopter" group trends quite unstable. If you're observing relatively constant treatment effects over time, especially after that first exposure, then you can be more confident that the difference-in-differences estimator is returning a sensible estimate of the average treatment effect on your treated units.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.