0
$\begingroup$

I am trying to see how a policy, say the treatment that happens in period 0, affects an outcome. I am interested in the time-varying effect of the policy. I have a panel data in long format, for example: | State | Treated | outcome | Period | | ---- | --------- | ------ | ------ | | 1 | 1 | 50 | -2 | | 1 | 1 | 50 | -1 | | 1 | 1 | 50 | 0 | | 1 | 1 | 55 | 1 | | 1 | 1 | 60 | 2 | | 1 | 1 | 70 | 3 | | 2 | 0 | 50 | -2 | | 2 | 0 | 50 | -1 | | 2 | 0 | 50 | 0 | | 2 | 0 | 50 | 1 | | 2 | 0 | 51 | 2 | | 2 | 0 | 52 | 3 |

etc

I saw different ways to perform the regression:

  1. Like in Cornwell and Cunningham (2013) with lag and lead year dummies, interactions and a dummy for the treatment: $$y_{st} = \beta_{0} + \beta_{1}\text{Treat}_{s} + \sum_{j \neq k} \lambda_{j} \text{Year}_{t=j} + \sum_{j \neq k} \delta_j \left( \text{Treat}_i \cdot \text{Year}_{t=j} \right) + X_{st}'\gamma + \epsilon_{st}.$$ http://business.baylor.edu/scott_cunningham/cunningham_and_cornwell.pdf
  2. Or use two way fixed-effect models only with lag and lead year dummies indicating that the given state was a given number of periods away from the event of interest in the respective time period like $$y_{s t}=\alpha+\sum_{j=2}^{J} \beta_{j}(\operatorname{Lag} j)_{s t}+\sum_{k=1}^{K} \eta_{k}(\operatorname{Lead} k)_{s t}+\mu_{s}+\lambda_{t}+X_{s t}^{\prime} \gamma+\varepsilon_{s t}$$ http://ftp.iza.org/dp13524.pdf
  3. Or like in Miller, et.al (2019) with interaction items and year dummies but not a dummy for the treatment $$Y_{s t}=\text { Treat }_{s} \times \sum_{y==-6 \atop y \neq-1}^{3} \beta_{y} I\left(t-t_{s}^{*}=y\right)+\beta_{t}+\beta_{s}+\gamma \mathbf{I}(j=t)+\epsilon_{s t}$$ http://www-personal.umich.edu/~mille/ACAMortality.pdf

My questions are:

  • What is the difference between the above models?
  • How should I interpret the results?
  • Which one should I choose?
  • And how should I see how the treatment intensity (from 1 to 3) will affect the outcome?

My programming language is R so I do appreciate any further hints on the codes but clarifying the specification questions is already a big favor to me.

I don't know why the table is shown in the preview but not the published page, you can find it here

$\endgroup$
3
  • $\begingroup$ Welcome. Does the policy affect all treated units at the same time, or was it rolled out in waves? $\endgroup$ Jul 14 at 0:12
  • 1
    $\begingroup$ Also, please provided links to the papers you cite. I think it's helpful to others reading your post. $\endgroup$ Jul 14 at 0:14
  • $\begingroup$ @ThomasBilach, yes at the same time and I added the refs $\endgroup$
    – xxx
    Jul 14 at 9:26
0
$\begingroup$

What is the difference between the above models?

In my opinion, all three models achieve the same objective. They're investigating time-varying treatment effects. In other words, they assess how effect evolve from time since exposure. And, if your subset of treated entities become exposed to a policy/law at the same time, then you should find all three approaches yield similar results.

The first model is an adaptation—not an exact reproduction—of an equation used in an article by Cornwell and CunningHam (2013). In equation (1) on page 14 they use the 'classical' difference-in-differences formulation to investigate the effect of early-repeal states (prior to Roe v. Wage) on sextually transmitted infections. The treatment/control dummy is the subset of states legalizing abortion pre-Roe. Table 2 reports coefficients on the interaction of a treatment/control dummy with a series of year indicators from 1985–2000. I believe they used 1985 as a reference. The effect for the year 2000 is dropped as well, but this is due to the inclusion of state-specific time trends. The summation notation isn't used but the difference-in-differences coefficient is $t$-subscripted (i.e., $\beta_{3t}$), suggesting the policy effects vary with time. The model you're porposing is very similar in structure. The summation suggests you're investigating the $j$-th additive treatment effect for all $j$ not equal to $k$. I assume $j$ denotes the individual time periods. Just ensure you tell your audience what $k$ represents in your setting. If it denotes one of the omitted periods, say so.

The second equation was reproduced from a discussion paper by Clarke and Schythe (2020) on event study designs more generally. In the basic formulation, $\operatorname{Lead} j$ and $\operatorname{Lag} j$ represent binary variables indicating that the given unit $s$ is a given number of periods away from the event of interest in the respective time period $t$. This equation is often used to investigate exposures that vary across $s$. In other words, some of the treated $s$ may enter into treatment early, while others become exposed much later. In any staggered adoption framework, there is no "time 0" around which to standardized the subset of controls. In fact, the set of units never exposed to the policy should equal 0 in all periods. In your setting, however, the policy affects treated $s$ at a precisely defined time period $t$, in which case we can infer the untreated would have adopted the policy at the same time as the treated. This is a very important to distinction. It's also worth noting that the lag indicators index the pre-treatment periods and lead indicators index the post-treatment periods. It doesn't really matter how you define them, just be clear about what epoch the leads/lags represent!

The third equation was reproduced from a working paper by Miller and colleagues (2019) which investigate the relationship between Medicaid enrollment and mortality. In their equation, the authors explicitly specify the number of leads and lags. Indicator variables $I\left(t-t_{s}^{*}=y\right)$ appear to measure the time relative to the implementation year, $t_{s}^{*}$, of the expansion in each state, and were consistently 0 in all periods for non-expansion states. The omitted period is $y = −1$, the year before the expansion. If mortality rates for expansion and non-expansion states were trending similarly prior to the expansion, then the estimated coefficients associated with event times $y = −6$ to $y = −2$ should be indistinguishable from 0. This is, yet again, a setting where the timing of expansions weren't uniform across $s$.

How should I interpret the results?

Technically, you interpret the model in the same way as any other difference-in-differences model. The only difference is now your effects should be interpreted as in reference to the omitted period.

Which one should I choose?

It's hard to say.

It's my understanding that your treatment of interest is standardized, affecting treated units at a single point in time. If so, then I'm partial to the 'classical' difference-in-differences specification (i.e., Eq. 1). Simply multiply your treatment/control dummy with time indicators for all periods before and after treatment. In fact, equations 1, 2, and 3 should yield similar point estimates, so long as the timing of the policy is the same for all treated $s$ in your sample. However, in settings with multiple groups starting (ending) treatment at different times, then you should opt to go with equation 2 or 3—not 1.

And how should I see how the treatment intensity (from 1 to 3) will affect the outcome?

Arrange the estimates in tabular form, or better yet, plot them. A nice visual can do a lot of talking for you.


Suggested Code

Don't make your life too difficult.

Assuming the data frame is as it looks in your post, simply interact treated with period and R will return a separate treatment effect in each period $t$. You don't even need to create the variable period. A year variable should do the trick.

### The classical DiD approach

# Interact `treated` with `period` or `treated` with `year`
# Ensure `period` or `year` is a factor variable
# You can change the reference period

lm(outcome ~ treated * period + covariates, data = ...)

### The classical approach with state and year fixed effects

# If the policy is adopted uniformly across states, then the estimates should remain unchanged
# The fixed effects absorb the treatment dummy and the individual period effects
# The interaction terms will remain

lm(outcome ~ as.factor(state) + as.factor(year) + treated * period + covariates, data = ...)
$\endgroup$
7
  • $\begingroup$ Thank you so much for your answer Thomas. Yep, I agree with you, with my setting, the outcomes from various models should be very similar. I initially started with the did package by Brantly Callaway and Pedro Sant'Anna and set the 'Group' of treatment to the same period of the intervention (I used att_gt and then aggte). And I then tried with a piece of code as the last line you use here where I used -1, the period right before the intervention as reference by convention and also add year dummies and respective interactinos for previous epochs to show the common trend. $\endgroup$
    – xxx
    Jul 15 at 16:28
  • $\begingroup$ Basically, I wish to have a plot as the pages 33 and 34 from Miller et al (2019). But the results turn to be very different from both ways and this is why I feel so confused..... $\endgroup$
    – xxx
    Jul 15 at 16:29
  • $\begingroup$ Why are your results different? How are the trends in the pre-period? $\endgroup$ Jul 15 at 16:40
  • $\begingroup$ I have no clue about the difference...The pre-coefficients are all insignificant... $\endgroup$
    – xxx
    Jul 15 at 16:56
  • $\begingroup$ That’s good news. So now you should plot the coefficients. The code I recommended should work. Follow-up with me if anything is still unclear. $\endgroup$ Jul 15 at 17:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.