3
$\begingroup$

My question relates to how the parallel trends assumption operates in a difference-in-difference (DID) analysis when there are two interventions of interest (in the plot, T1 and T2). T1 and T2 are policy changes that can both be in effect at the same time (they affect the same outcome via different mechanisms). T2 is implemented while T1 is already in effect.

In a DID analysis with a single intervention, there is one pre-intervention period and one post-intervention period. The trends in the outcome for the control and treatment groups must be parallel in the pre-intervention period for the parallel trends assumption to hold.

This is also the case in a DID analysis with two interventions: the trend in the outcome for the control and treatment groups must be parallel in the period prior to the first intervention (T1). However, assuming there is a treatment effect, the slope for the treatment group will change after T1 (shown by the solid green line in the plot; the dotted line is the counterfactual). Now, the parallel trends assumption is not met for the pre-intervention period before intervention T2 (i.e. the trends are not parallel for the time between T1 and T2). And if this assumption is not met, then we cannot perform DID analysis to estimate the effect of intervention T2.

I know that DID analyses with multiple interventions can and have been performed, but I must be misunderstanding something about the parallel trends assumption. How can the parallel trends assumption be met so that we can estimate the effect of intervention T2?

enter image description here

$\endgroup$
2
  • $\begingroup$ You can have multiple treatments. But, what is the second treatment here? In your example, is a new treatment implemented (e.g., T2) while the first treatment is already in effect? Or, is T2 when the control group enters into treatment. Just curious because sometimes treatment is staggered and units enter into treatment at different times. $\endgroup$ Commented May 8, 2020 at 3:33
  • $\begingroup$ T1 and T2 are policy changes that can both be in effect at the same time (they affect the same outcome via different mechanisms). So yes, T2 is implemented while T1 is already in effect. Both T1 and T2 only affect the treatment group. $\endgroup$
    – Bicep
    Commented May 8, 2020 at 4:07

1 Answer 1

1
$\begingroup$

In theory, it will be difficult to support claims of trend equivalence if your purpose is to assess the causal impact of intervention 1 and intervention 2—separately. As you note, we often rely on heuristic procedures when assessing parallel trends in a difference-in-differences (DiD) framework. Plotting the evolution of the group trends is merely a visual aid. In the setting sketched out in your question, the average of the outcome for units comprising the control group are used to assess the counterfactual trend for both interventions.

Visually, the validity of the 'parallel paths' assumption is supported before the first intervention (i.e., T1). However, intervention 1 sets treated units on a different growth trajectory after its effective date. If the trend in the control group remains constant, as indicated in your plot, then this counterfactual trend is not a suitable assessment of what would have happened in the absence of the second treatment. Counterfactual units were ineligible for both treatments. Effects may be overstated since the pre-period of the second treatment (which is the post-period of first treatment) is already trending upward.

Often times, there is a withdrawal of treatment which gives effects time to dissipate before a second intervention goes into effect. But in your setting, a second intervention, which is a qualitatively different treatment, commences while the first is already in place.

How can the parallel trends assumption be met so that we can estimate the effect of intervention T2?

There is no easy solution if there is a large differential growth trajectory due to the implementation of the first intervention. One solution is to find more units (e.g., states, counties, firms, individuals, etc.) not subjected to either intervention. Based upon your question, you do not make clear how many units comprise the control group. With many units, you could partition your control group into subgroups and try to assess parallelism with a subset of never treated units. I would be curious to know if the second intervention affects all of the treated units from the first intervention. Arguably, there may have been control units that were more likely to be eligible for the second intervention but did not receive it for some reason.

Another concern is the number of $t$ periods before treatment 2 goes into effect. It is methodologically untenable to assess parallelism between two groups with scanty pre-treatment observations (e.g., $t^{Pre} < 3$). Assume $t_{0}$ is the first post-treatment period since adoption of the first intervention. Then by design, $t_{0}$ is also the first (and possible only) pre-treatment period before the second intervention arrives at $t_{0} + 1$. In this case, it would be hard to conduct separate DiD analyses without sufficient pre-treatment observations before the second intervention.

Lastly, you note the second treatment is a different type of intervention, but affects the outcome via a different mechanism. How different is the follow-up policy? Could control units just as easily have received intervention 2 first? Was the second policy adopted to intensify the effects of the first treatment? These are important substantive considerations and require intimate knowledge of the treatment and its effects on your outcome. In sum, it may be difficult to sell to a reader why you conducted a separate DiD analysis for the second treatment when the pre-period trend is likely less stable (more volatile) due to the first phase of treatment.

You could try conducting one DiD analysis, interacting a treatment dummy with separate post-treatment indicators (i.e., separate post-treatment period dummies). You could assess treatment effects in different treatment phases. I would be curious to see how much the first policy is responsible for the change in your outcome.

I hope this helps!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.