1
$\begingroup$

In reading this paper of Bacon,2018 , I saw a paragraph

I also explain why the negative weights occur: when already-treated units act as controls, changes in their treatment effects over time get subtracted from the DD estimate. This negative weighting only arises when treatment effects vary over time, in which case it typically biases regression DD estimates away from the sign of the true treatment effect

How come an already-treated units act as control? It should be only from the "reverse treatment", for example, a country pass the law and then cancel the law later in a multiple treatments and periods, rather than the reason in the last sentence above. I understand "treatment effects vary overtime" that, in a specific country, the first year of law adoption, the treatment effect is stronger than 5 years later.

$\endgroup$
7
$\begingroup$

Essentially what happens is that the staggered DID can be interpreted as a weighted average of two-period DIDs. So if a group is treated in multiple periods and another is not it could happen that the treated group enters in as a control. This matters if there is treatment effect heterogeneity (for example the treatment has a larger effect in the 2nd active period than in the first) because it could bias the end results. I give more details about how this happens below.


Here is a more in-depth look at how and why this happens. For a more complete view, you should definitely read the 2020 paper I cite.

First, let us discuss why this occurs. Consider a two-way fixed effects estimator of the ATE via the regression formulation,

$$Y_{i,g,t} = \alpha_g + \alpha_t + \beta D_{g,t}+e_{igt}$$

It can be shown (see Theorem 1 of de Chaisemartin and D'Haultfoeuille 2020) that under parallel trends we can write the ATE as,

$$\beta = \mathbb{E}{\Large{[}}\sum_{(g,t);D_{g,t}=1}\frac{N_{g,t}}{N_1}w_{g,t}\Delta_{g,t}\Large{]}$$

where $\Delta_{g,t}$ denotes the ATE for a specific group-period and $w_{g,t}$ denotes the weights. The paper you referenced studies a special case of this estimator called a "staggered adoption" DID.

In this setting, Bacon (2018) gives an intuitive explanation for why negative weights occur,

I also explain why the negative weights occur: when already-treated units act as controls, changes in their treatment effects over time get subtracted from the DD estimate.

So to understand this phenomenon we need to understand why negative weights occur. The 2020 paper in fact presents a simple example of how this happens in staggered adoption,

Group 1 is untreated at periods 1 and 2 and treated at period 3, while group 2 is untreated at period 1 and treated both at periods 2 and 3.

With some additional assumptions in the paper it is shown that,

$$\beta = 0.5 \mathbb{E}[\Delta_{1,3}]+\mathbb{E}[\Delta_{2,2}]-0.5\mathbb{E}[\Delta_{2,3}]$$

Note that the weights are not equal and the weight assigned to group 2 period 3 is negative even though that group is treated in that period. You can read their paper for more details about how this can badly bias the results.

However, going back to the main question at hand we want to understand why this third weight is negative. If we have a mathematical understanding of this we can understand what is meant by treated units acting as controls. In particular, this example is really the average of two DIDs.

The first two-period design (remember in period 2 only group 2 is treated): $$\mathbb{E}[Y_{2,2}-Y_{2,1}-(Y_{1,2}-Y_{1,1})]$$

Second two-period design (now both groups are treated): $$\mathbb{E}[Y_{1,3}-Y_{1,2}-(Y_{2,3}-Y_{2,2})]$$

But notice, group 2 is treated in both periods! And, since the second two-period design needs to study the evolution of the treatment effect from period 2 to period 3 we have effectively used $Y_{2,2}$ a treated unit as a control!

Now let us dig in a little more. Notice that as Bacon asserts this only happens when treatment is unstable, i.e. it changes over time. Why is this? Well under common trends the second DID could be written as,

$$\mathbb{E}[\Delta_{1,3}-(\Delta_{2,3}-\Delta_{2,2})]$$

However, if treatment is homogenous then we get, $\mathbb{E}[(\Delta_{2,3}-\Delta_{2,2})]=0$. Thus, this issue does not arise.

A little literature review:

In general, these kinds of biases are quite common when estimating the ATE. Notice, that Imbens and Angrist (1994) show a very similar result that the IV estimator of the ATE can be written as a weighted average. Indeed, the famous monotonicity condition that they employ to deal with this ensures that we do not get negative weights. However, this makes their recovered estimate of the ATE "local". This is because the estimate can now only be considered an effect on "compliers". Of course, the monotonicity condition can be shown to be equivalent to a selection model (Vytlacil 2002) and this equivalence has allowed for a deeper study of heterogeneity in treatment effects (see Heckman and Vytlacil 2005). Thus, this kind of logic has actually come up in several places in the econometric literature.

$\endgroup$
6
  • 6
    $\begingroup$ (+1) Great answer, @Ariel. $\endgroup$ Jun 5 at 18:45
  • $\begingroup$ Thanks so much, Ariel, I had some questions as below: (1) "So if a group is treated in multiple periods and another is not it could happen that the treated group enters in as a control" => Is the group a country in my case? And, how come a group can be treated in multiple periods can be control in this sentence because the another will only be control all the time. Or do you mean the "group treated" in your sentence is control in some particular period for other treated groups that you did not mention in the sentence above? $\endgroup$ Jun 6 at 23:23
  • 3
    $\begingroup$ Yes - a group could be a country if the treatment is at the country level. The reason for this notation is to allow extra flexibility. Namely, an individual-level panel where the group could be something like nationality if the treatment was done at that level. For question 2, what happens is that a group that is treated in the previous period could act as a control for treated groups in the next period. If you look at that example again, the second 2-period DID has group 1 period 3 as treated and the difference between group 2 period 3 and group 2 period 2 as control. $\endgroup$
    – Ariel
    Jun 7 at 0:22
  • $\begingroup$ Regarding $N_1$, it is the number of treated units (de Chaisemartin and D'Haultfoeuille 2020, p.2969), therefore the $N_1$=6 in this simple example, so I am doubting if $$\beta = 1/6 \mathbb{E}[\Delta_{1,3}]+ 1/3\mathbb{E}[\Delta_{2,2}]- 1/6\mathbb{E}[\Delta_{2,3}]$$ . From my point of vie, it should relate to $w_{g,t}$ but I do not know how to relate them $\endgroup$ Jun 7 at 5:53
  • 1
    $\begingroup$ The paper defines $N_1=\sum_{i,g,t} D_{i,g.t}$ and does denote the number of treated units. The coefficients are given by $\frac{N_{g,t}}{N_1}w_{g,t}$ since in this example $N_{g,t}/N_1$ does not vary you can apply Proposition 1 to get the weights. If you are unsure about these results, I suggest you attempt to replicate the proofs of Proposition 1 and Theorem 1. $\endgroup$
    – Ariel
    Jun 7 at 13:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.