# Refining a difference-in-difference analysis

I'm doing a difference-in-differences analysis with one pre-treatment time (0), and two post-treatment time points (1,2).

My basic regression model is:

= $$β_0+β_1T_1 +β_2T_2 +β_3S+ β_4(S∗T_1)+β_5(S∗T_2)+ε$$

where $$T_1$$ is a dummy (equals 1 for time 1, 0 otherwise) $$T_2$$ is a dummy (equals 2 for time 2, 0 otherwise) $$S$$ is a dummy (equals 1 for treatment group, 0 otherwise)

The DiD coefficient is $$β_4$$ for the first post period, and $$β_5$$ for the second period.

However, in $$T_2$$ a second policy affected a subset of the treatment group ($$S$$). I'd like to isolate the effect of this separate policy from the main policy, by comparing the affected individuals in the treatment group to the unaffected individuals from the treatment group (so basically, a DiD analysis within the original treatment group). The second policy did not affect the original control group.

Would this be the correct regression model for analysis:

$$= β_0+β_1T_1 +β_2T_2 +β_3S_1+ β_4(S_1∗T_1)+β_5(S_1∗T_2)+β_6(S_2∗T_2)+ε$$

where $$β_6$$ is the DiD coefficient comparing the affected individuals and unaffected individuals (from the original treatment group) between the second 2 time points, and:

$$S_1$$ is a dummy variable that equals 1 if the individual is in the treatment group for the first policy, and

$$S_2$$ is a dummy variable which equals 1 only if the individual is in the subset of the treated group that is affected by the second policy.

Many thanks.

You state: "I'd like to isolate the effect of this separate policy from the main policy, by comparing the affected individuals in the treatment group to the unaffected individuals from the treatment group (so basically, a DiD analysis within the original treatment group). The second policy did not affect the original control group."

(2) $$Y_i= β_0+β_1T_1 +β_2T_2 +β_3S_1+ β_4(S_1∗T_1)+β_5(S_1∗T_2)+β_6(S_2∗T_2)+ε$$"

[to change] You should not forget to have a coefficient $$\beta_7 S_2$$, otherwise eventual differences between $$S_1$$ and $$S_2$$ could drive your $$\beta_6$$.

[robustness test?] Moreover, you might want to test (in a separate regressions) that there is no heterogeneous effect of the first policy on subgroup $$S_2$$ in period $$T1$$. That could also drive your $$\beta_6$$ in period $$T2$$.

[robustness test?] Note also that in the end, the unaffected group for both policies is still somehow in the control group when you compute $$\beta_6$$ (for instance, its members contribute to the determination of $$\beta_2$$). A "DID analysis within the original treatment group" would basically be (on the subset $$S1$$):

(3) $$Y_i= \gamma_0+\gamma_1T_1 +\gamma_2T_2 +\gamma_3S_2+ \gamma_4(S_2∗T_2)+ε$$"

In theory, your $$\beta_6$$ should compare only those affected by policy 2 in the second time period to (1) all those never affected and (2) those in S2 in the previous time periods. So that basically seems like what you want to measure. You might want to include an S2 dummy to take care of baseline differences between the S2 treatment and control.

This is not a complete answer - I'm still not confident this model is correct. For example, I might worry about the acceptability of the control group for S2 (by default, S1 + original control). Did you consider modeling the two policies separately?

• Thanks Alexandre. Yes, I can model the two policies separately (and this gets me the result I wanted). I do one model to test the effect of policy 1, by comparing those affected only by policy 1 to the control group. – hassapikos Aug 21 '19 at 11:19
• Then I do a model to isolate/quantify the effect of policy 2, by doing a DiD comparing the those affected by policy 2 (and 1), to those only affected by policy 1. I don't think I need to compare those affected by policy 2 to the original control in order to do that, if that affects your suggested solution? – hassapikos Aug 21 '19 at 13:04