Skip to main content
2 of 16
Further explication to understand why this is important.
Thomas Bilach
  • 6.5k
  • 2
  • 13
  • 33

To BruceET's point, you want to do everything you can to show that the intervention is plausibly un-confounded. In my opinion, difference-in-differences (DiD) can work in this setting. See here for a gentle introduction to the DiD methodology. I will address some caveats with your study later.

One group has always had a "treatment" throughout the time period (always had a specific legal prohibition). One group never had the "treatment" within the time period (never had the prohibition). One group had the "treatment", but each member of the group had the "treatment" at a different time within the period (individual states enacted prohibition in different years).

First, we should get your empirical model correct. I should note, however, that you cannot proceed with the "classical" DiD approach. You do not have well-defined pre-/post-treatment periods. Treatment only affects a subset of U.S. states, which gives you a group of "non-adopter" states to serve as a control group. Treated state jurisdictions enter into treatment at different times, a subset of which have only one treatment history (i.e., the "always" treated). In staggered adoption settings, you are outside the realm of the "classical" approach. Instead, you must use with the "generalized" approach, which is also a two-way fixed effects estimator. If I understand your question, you wish to estimate the following:

$$ \text{Outcome}_{st} = \alpha + \gamma_{s} + \lambda_{t} + \delta \text{Prohibition}_{st} + \epsilon_{it}, $$

where you observe a continuous outcome in state $s$ in year $t$. $\gamma_{s}$ and $\lambda_{t}$ denote fixed effects for states and years, respectively. The variable $\text{Prohibition}_{st}$ is your policy (i.e., treatment) dummy.

Quoting from the comments:

So I test a model like "outcome ~ year + policy + year x policy" where the "year x policy" term is the DiD term and "policy" is a binomial variable indicating each year whether or not the specific state has the specific policy?

Not quite right. You need to include state and year effects. In addition, your interaction should instantiate the policy variable appropriately. I recommend creating the policy variable manually. It is still your interaction term, you just need to define it in a different way as your "post-treatment" periods vary across states. Again, your policy variable, $\text{Prohibition}_{st}$ is a dummy equal 1 in all years $t$ when state $s$ adopts the policy, 0 otherwise. The "never-receivers" will be "always 0" for the entire observation period. The "always treated" will be equal to 1 for the entire panel. In R, your model should look something like this:

outcome ~ lm(as.factor(state) + as.factor(year) + prohibition + ..., data = ...)

In general, the policy dummy is allowed to take any pattern, which is something noted in Jeffrey Wooldridge's text Introductory Econometrics: A Modern Approach (7th Edition). The estimate of $\delta$ is your treatment effect.

The pattern of the dummy matters with respect to identification, though. See this post where the pattern of the treatment dummy is addressed in detail. The issue is the "always treated" jurisdictions. It will be difficult—actually impossible—to demonstrate (visually) common trends without sufficient "pre-event" data. You could circumvent this problem by excluding the "always treated" from your analysis. To Dimitriy's point (see comments), this severely limits your sample, leaving you with approximately 30 states, 8 of which will be treated as some point between 1984 and 2018. You will also have to look into some finite-sample adjustment to deal with your standard errors.

I don't know of any valid methods for dealing with the "always treated" (i.e., always equal to 1) states in these settings. With respect to the staggered adoption states, you could center your states on the official adoption year and assess parallelism of the group trends. If possible, try and get more pre-event data on the "always treated" states. Or, maybe you could subject the "always treated" to a separate analysis. Do these states repeal any of the prohibition policies over time? This might be something to consider.

You might also find this recent working paper interesting.

Thomas Bilach
  • 6.5k
  • 2
  • 13
  • 33