Can this additional data improve the validity of Difference-in-Differences estimate

Question

I am aiming to measure the impact of a treatment (a marketing stimulus) on product revenue. The data has resulted from a natural experiment. The set-up is tabulated below. Cells in the table indicate whether customers in a group received marketing stimulus or not.

Customer group	Sales Quarter 1	Sales Quarter 2
Control group	No	No
Treatment Group 1	No	Yes

Because there is customer level data, I plan to estimate using a panel regression model with revenue in a quarter as Y and the following variables as Xs (I may include additional variables if the trends in TG & CG are different and unrelated to the treatment effects):

1. Dummy variable for whether customer revenue was measured in Quarter 1/ Quarter 2
2. Dummy variable for whether customer was in the treatment group/ control
3. The interaction between the two dummy variables, the co-efficient of which would be the Difference-in-Differences estimate, or the estimate of the treatment effect.

I stated the above to set context. My query however, is about an additional set of data that is available, indicated as treatment group 2 in the table below:

Customer group	Sales Quarter 1	Sales Quarter 2
Control group	No	No
Treatment Group 1	No	Yes
Treatment Group 2	Yes	No

My question is whether this information can be used to improve the validity of the study and if so, how?

Answer 1

The problem with estimating the effects of both treatments in one equation is the inclusion of the time dummies separating pre- versus post-treatment. The pre-treatment period for group 1 is, technically, the post-treatment period for group 2.

Let's try to formalize this by writing out your difference-in-differences equation. Here is the canonical specification with two treatment groups.

$$ y_{it} = \alpha + \gamma_1 T^{1}_{i} + \gamma_2 T^{2}_{i} + \lambda_1 B_{t} + \lambda_2 A_{t} + \delta_1 (T^{1}_{i} \times B_{t}) + \delta_2 (T^{2}_{i} \times A_{t} ) + \epsilon_{it}, $$

• $T^{1}_{i}$ = 1 for individuals in Treatment Group 1, 0 otherwise
• $T^{2}_{i}$ = 1 for individuals in Treatment Group 2, 0 otherwise
• $B_{t}$ = 1 in Quarter 1, 0 otherwise (i.e., "before" period)
• $A_{t}$ = 1 in Quarter 2, 0 otherwise (i.e., "after" period)

The redundancies should be evident. Note how $B_{t}$ and $A_{t}$ index all epochs. One time indicator must be dropped to avoid collinearity. If the pre-period is dropped, which will likely be the case if estimated in the order presented above, then software will only return an estimate for $\delta_2$.

In my opinion, I would run two separate equations. The first would only include $T^{1}_{i}$ and your controls; the second would only include $T^{2}_{i}$ and your controls. The latter equation is the effect of withdrawing treatment (i.e., switching from 1 to 0). It's rare to observe entities starting out in a treated condition, but the equation is estimable.

Technically, it doesn't matter which epoch we set as a reference. In most cases, the absolute value of the treatment effect is the same either way, they're just opposite in sign. This is true in general. However, symmetry isn't always guaranteed. Say revenue was log-transformed. Here, the effect of moving into treatment (i.e., switching from 0 to 1) isn't the same as moving out (i.e., switching from 1 to 0).

In short, including multiple treatment groups into one model isn't appropriate. The second group of individuals (i.e., $T^{2}_{i}$) don't have any pre-event data. In my opinion, I would subject the customers whose marketing stimulus was removed to a separate analysis.