Skip to main content
6 of 16
Further explication to understand why this is important.
Thomas Bilach
  • 6.5k
  • 2
  • 13
  • 33

To BruceET's point, you want to do everything you can to show that the intervention is plausibly un-confounded. In my opinion, difference-in-differences (DiD) can work in this setting. See here for a gentle introduction to the DiD methodology. I will address some caveats with your study later.

One group has always had a "treatment" throughout the time period (always had a specific legal prohibition). One group never had the "treatment" within the time period (never had the prohibition). One group had the "treatment", but each member of the group had the "treatment" at a different time within the period (individual states enacted prohibition in different years).

First, we should get your empirical model correct. I should note, however, that you cannot proceed with the "classical" DiD approach. You do not have well-defined pre-/post-treatment periods. Treatment only affects a subset of U.S. states, which gives you a group of "non-adopter" states to serve as a control group. Treated state jurisdictions enter into treatment at different times, a subset of which have only one treatment history (i.e., the "always" treated). In staggered adoption settings, you are outside the realm of the "classical" approach. Instead, you must use with the "generalized" approach, which is also a two-way fixed effects estimator. If I understand your question, you wish to estimate the following:

$$ \text{Outcome}_{st} = \alpha + \gamma_{s} + \lambda_{t} + \delta \text{Prohibition}_{st} + \epsilon_{st}, $$

where you observe a continuous outcome in state $s$ in year $t$. $\gamma_{s}$ and $\lambda_{t}$ denote fixed effects for states and years, respectively. The variable $\text{Prohibition}_{st}$ is your policy (i.e., treatment) dummy.

Quoting from the comments:

So I test a model like "outcome ~ year + policy + year x policy" where the "year x policy" term is the DiD term and "policy" is a binomial variable indicating each year whether or not the specific state has the specific policy?

Not quite right. You need to include state and year effects. In addition, your interaction should instantiate the policy variable appropriately. I recommend creating the policy variable manually. It is still your interaction term, you just need to define it in a different way as your "post-treatment" periods vary across states. Again, your policy variable, $\text{Prohibition}_{st}$, is a dummy equal to 1 in precisely the state-adoption years, 0 otherwise. The "never treated" states will be "always 0" for the entire observation period. Similarly, the "always treated" states will be "always 1" in all state-years. In R, your model should look something like this:

model <- lm(outcome ~ as.factor(state) + as.factor(year) + prohibition + ..., data = ...)

In general, the policy dummy is allowed to take any pattern, which is something noted in Jeffrey Wooldridge's text Introductory Econometrics: A Modern Approach (7th Edition). Your estimate of $\delta$ is your treatment effect.

It should be noted, though, that the pattern of the policy dummy matters in DiD settings. See this post where the pattern of the treatment dummy is addressed in detail. The main issue is the "always treated" jurisdictions. It will be difficult—actually impossible—to demonstrate (visually) common trends without sufficient "pre-event" data. You could circumvent this issue by excluding the "always treated" from your analysis. To Dimitriy's point (see comments), this severely limits your sample, leaving you with approximately 30 states, 8 of which will be treated as some point between 1984-2018. You will also have to look into some finite-sample adjustments to deal with your standard errors.

I don't know of any valid methods for dealing with the "always treated" (i.e., always equal to 1) states in these settings. It will be difficult to make claims regarding the inter-temporal evolution of the "always treated" state trends relative to the "never treated" state trends (or soon to be treated state trends) before the policy goes into effect—because you never observe them (i.e., the "always treated") pre-policy! A surrogate measure might help if it allows to obtain more pre-event data and you believe treatment affects the surrogate outcome in the same way.

With respect to the staggered adoption states, you could center your states on the official adoption year and assess parallelism of the group trends. If possible, try and obtain more pre-policy data on the "always treated" states. Or, maybe you could subject the "always treated" to a separate analysis. Do these states repeal any of the prohibition policies over time? This might be something to consider.

You might also find this recent working paper interesting.

Thomas Bilach
  • 6.5k
  • 2
  • 13
  • 33