For my master thesis I am running a set of difference-in-differences (DD) estimators and lately I am a bit confused about the correct specification. I am investigating the differential impact of the COVID-19 crisis on minority unemployment in the US. I am using monthly Current Population Survey (CPS) data and additionally I used data about the stringency of COVID-19 policies across US States. Therefore, I am using a DD estimator for all months post-COVID-19 (up to April 2021). The specification I have right now looks like this:

$$ y_{ismt} = \sum_{i=1}^{n=13} \eta_n COVIDmonth + \beta_2 race_i + \beta_3 index\_level_{smt} + \sum_{i=1}^{n=13}\theta_n (COVIDmonth\times race_i) + \sum_{i=1}^{n=13}\nu_n (race_i\times index\_level_{smt}\times COVIDmonth) + \beta_5 X_{ismt} + \beta_6 covid\_cases_{istm-1}+ \gamma_s + \omega_{mt} + \epsilon_{ismt} $$

where where $y_{ismt}$ is the labor market outcome for an individual $i$ in state $s$ in a given month $m$ and year $t$, $COVIDmonth$ is a dummy for each pandemic month, $race_i$ is an individual's race, $index_level$ are the total days spent at a certain index level (example: in October, an individual in the State of Iowa was exposed to an index level above 60 for 70 days), $covid_cases_{istm-1}$ is a measure of the one month lagged covid cases, $X_{ismt}$ is a vector of controls, including socio-demographic variables as age, a second degree polynomial of age, gender, educational level and martial status, as well as major occupation and industry codes, and an indicator of whether the industry was classified as essential during the pandemic, $\gamma_s$ are state-fixed effects and $\omega_{mt}$ are year-of-month fixed effects. Standard errors are clustered at the state level.

To my question: My intuition would says to me that like in the conventional DiD framework I have to include all month dummies (i.e., post), races (treatment group, white = control), and the treatment (i.e., index_level). Then I have to interact month and race, to disentangle the effects of the month itself, and the difference-in-difference-in-differences (DDD) estimator delivers the differential impact of the stringency exposure between whites and non-whites. However, as I said, I am pretty confused if this is correct. I also thought that maybe this approach is more correct:

$$ y_{ismt} = \sum_{i=1}^{n=13} \eta_n COVIDmonth \times \beta_3 index\_level_{smt} + \beta_2 race_i +\sum_{i=1}^{n=13}\nu_n (race_i\times index\_level_{smt}\times COVIDmonth ) + \beta_5 X_{ismt} + \beta_6 covid\_cases_{istm-1}+ \gamma_s + \omega_{mt} + \epsilon_{ismt} $$

such that the treatment = COVIDmonth x index_level.

Any help is highly appreciated! Thank you very much!

  • $\begingroup$ Welcome. Is the treatment when the index surpasses 60, in which case it’s binary (above/below), or is it the number of days (continuous) individual $i$ is exposed at a particular index level? $\endgroup$ Jul 1 at 18:12
  • $\begingroup$ Hello and thank you for your reply! The treatment is continuous, so it is the accumulated number of days exposed to a level > 60 at the 12th of each month (the reference week of the CPS.) The variation across states in each month is used to investigate whether longer exposure has a significant impact on minority unemployment. $\endgroup$
    – R_hub
    Jul 1 at 19:43
  • $\begingroup$ So the “time to exposure” varies across $i$? I only ask because you may have to define the treatment variable manually to account for this. $\endgroup$ Jul 1 at 19:54
  • $\begingroup$ The time of exposure varies across states. For instance: 12th Juli 2020: an Arizona citizen was exposed 80 days at a stringency level above 60. If policies continue, in August it may be 100 days and so on. $\endgroup$
    – R_hub
    Jul 1 at 19:58
  • $\begingroup$ I apologize. I meant across $s$, yes. So to be clear, individuals in state $s$ start as 0 in the months where the stringency level is 60 or below. Then, at some time $t$ when stringency levels exceed 60 the variable takes on positive integer values denoting the exposure days. Is this accurate? $\endgroup$ Jul 1 at 22:32

My intuition would says to me that like in the conventional DD framework I have to include all month dummies (i.e., post), races (treatment group, white = control), and the treatment (i.e., index_level).

First, you cannot proceed with the "conventional" or "classical" method since the 'time to exposure' varies across states. What you're proposing is actually a difference-in-difference-in-differences (DDD) estimator with staggered exposure epochs. It's more commonly referenced in economics as a triple difference (TD) estimator, as least as suggested here.

Second, it appears you're making a claim that the month dummies (i.e., month-year effects) index the post-treatment periods, but this is inaccurate. The period dummies do not demarcate pre- versus post-treatment; they simply model the time shocks. On the other hand, a variable labeled "post" or "after" would simply 'turn on' (i.e., switch from 0 to 1) in the month-years when state $s$ is above the cutoff, 0 otherwise. But note how "post-exposure" has no well-defined meaning in this context. The 'time to exposure' varies across $s$, which makes inferring a non-treated state's post-period a bit tricky.

If your model was truly a "conventional" TD setup, then you'd have a dummy denoting group membership (e.g., treatment/control state), a dummy denoting a sub-group within states that is more sensitive to the policy (e.g., white/non-white), and an indicator denoting the two discrete time periods (i.e., pre-/post-treatment). In your setting, however, the 'time to exposure' varies across $s$ which, in turn, affects the cumulative number of days in a treated condition. Thus, the duration of exposure for any individual $i$ depends upon when the state exceeds a particular stringency index. Due to the different timing impacts, we cannot distinguish between pre- versus post-treatment. We must define the interaction term in a different way.

Then I have to interact month and race, to disentangle the effects of the month itself, and the DDD estimator delivers the differential impact of the stringency exposure between whites and non-whites.

Not quite.

In your setting, you have three dimensions over which treatment may vary: state $s$, month-year $t$, and racial group $g$. You suspect non-whites living within states with restrictive COVID-19 policies are more sensitive to the exposure than others, hence why you're proceeding with a TD approach.

  • A quick word on notation. I would avoid using two distinct subscripts to denote time. I would just use $t$ and note in your paper that it's denoting all month-years in the sample. I say "month-year" because I assume you're working with 14 months of data. You must create some concatenated version of month and year to estimate the time effects. Thus, a month-year variable (e.g., March-2020, April-2020, May-2020,..., April-2021) seems appropriate.

The general representation of the TD equation would specify all second-order interactions terms and the continuous policy variable. It would look something like the following:

$$ y_{igst} = \gamma_{st} + \lambda_{gt} + \eta_{gs} + \delta COVID^{I>60}_{gst} + X_{igst}'\beta + \theta Cases_{s,t-1} + u_{igst}, $$

which includes state times month-year interactions (i.e., $\gamma_{st}$), group times month-year interactions (i.e., $\lambda_{gt}$), and group times state interactions (i.e., $\eta_{gs}$). $COVID^{I>60}_{gst}$ is the continuous policy variable, just defined in a different way. Here, we instantiate the triple interaction term manually to account for the staggered onset of exposure. Thus, $COVID^{I>60}_{gst}$ will start as 0 in all states in the early month-years of the pandemic, but at some time $t$ it will take on positive integer values denoting the cumulative days of exposure. In other words, the policy variable takes on its appropriate intensity/dosage when three conditions are satisfied: (1) the state is treated (i.e., $I>60$), (2) the individual/group is non-white, and (3) it is a post-treatment month. Again, the "timing" of the policy isn't well-defined, which means there's no easy way for software to create the policy variable for you. We cannot assign a single post-period when the start (end) of the exposure epochs vary across $s$. I would adapt this equation to suit your needs.


After reading your comments, I think I may have misadvised you. I will keep the previous response (see above) unless it's inapplicable and doesn't offer any value.

So it appears your measure of 'bite' or 'dosage' is time-invariant; it varies across $s$ but not over $t$. To be precise, it denotes the cumulative number of days exposed to a severe COVID-19 intervention. Let's call the policy/mandate a stay-at-home (SAH) order. Here is one way to proceed:

$$ y_{ist} = \gamma_{s} + \lambda_{t} + \delta (SAH_{s} \cdot P_t) + X_{ist}'\beta + \theta Cases_{s,t-1} + u_{ist}, $$

where $\gamma_{s}$ and $\lambda_{t}$ denote fixed effects for states and months, respectively. The intensity variable $SAH_{s}$ is the total number of days a state was in the severe policy condition. $P_t$ is a post-treatment indicator equal to unity in all months after the policy takes effect, 0 otherwise. The interaction of the constituent terms will return the policy variable variable of interest. Thus, the actual treatment variable should equal 0 in the months before the intervention, after which it should take on positive integer values to denote the cumulative days exposed to a stringent policy. Again, my answer assumes the SAH policy had not been implemented in any month, say, before April.

  • Note, I explicitly excluded the main effects for $SAH_{s}$ and $P_{t}$. The fixed effects will absorb the constituent terms. The model is not misspecified, as the relevant information is captured by the corresponding fixed effects.

In my previous answer, when I stated that the treatment varies across $s$, I was referring to the timing of each intervention. For example, suppose you were interested in the effect of school closures on scholastic achievement in U.S. counties. Now suppose the mandate was implemented in different months. Some county officials ordered closures in April, while others waited until July. In this setting, $P_t$ is not clearly defined, in which case we should instantiate the interaction term manually. But this doesn't appear to be your situation. The executive order(s) you're considering was enacted after March, so we can justify the use of a post-treatment variable to delineate the before-and-after epoch.

Now say you suspect a differential response to social distancing dictates across different sub-groups. For example, do you suspect severe mitigation strategies reduce the earnings of individuals employed in non-essential industries more than those in essential industries? If so, we can model this so long as we have data at the $i$ level. All you must do is introduce a third variable delineating the more sensitive sub-group (e.g., essential/non-essential, white/non-white, educated/uneducated, male/female, etc.). Let's call the third variable $S_i$ for the sensitive group. Here is one way to proceed:

$$ y_{ist} = \gamma_{s} + \lambda_{t} + \delta (SAH_{s}\cdot P_t) + \mu (S_i \cdot SAH_{s} \cdot P_t) + \zeta S_i + X_{ist}'\beta + \theta Cases_{s,t-1} + u_{ist}. $$

In this model, $\delta$ is capturing the effect of additional days of policy exposure; $\mu$ is capturing a differential response for the sensitive sub-group (e.g., non-essential employees). Given the granularity in your data, it's permissible to allow treatment effects to differ by industry, occupation, race/ethnicity, age, education, et cetera. Just don't go too crazy.

The equation doesn't quite tell the whole story. Some of the second-order interaction terms should appear in your output. For example, we can't ignore $S_i \times SAH_s$ and $S_i \times P_t$. In the paper you referenced, the authors do not explicitly specify them. I don't know why. I don't see any of the second-order terms in their tables, though I imagine they were estimated. Usually the estimates on the lower-order terms aren't of substantive interest.

Lastly, if the intensity variable (i.e., $SAH_s$) is 0 in non-adopter states and some integer value (i.e., days exposed) for treated states, then it retains, to some degree, a binary structure, thus we can use it to replace a treatment indicator (i.e., treatment/control dummy) in the simple two-group/two-period case.

I recommended some R code below. The variable essential is equal to 1 if the individual is employed in an essential industry, 0 otherwise. It's simply a placeholder. The two models should return identical estimates.

## Equation 1

# Includes state and month-year fixed effects
# They will absorb `sah` and `post`

lm(y ~ as.factor(state) + as.factor(month_year) + essential*sah*post + covariates, data = ...)

## Equation 2

# Three-way interaction term (no fixed effects)
# The main effects and second-order interactions will be estimated for free
# This approach doesn't result in any terms being dropped

lm(y ~ essential*sah*post + covariates, data = ...)
  • $\begingroup$ Thanks so much for your time and the response! I think my description of what I actually want to measure was inaccurate. The basic idea follows the approach of Gupta et. al (2020) link, so that the treatment is a continuous (dosage) measure of how long a policy has been in place. In contrast to their paper, I am using an Index which contains multiple policies (School Closings, Workplace Closings, Stay at home orders etc.) and I measure the effect for multiple months (they only do it for April 2020). The data is a cross section, I observe States over time. $\endgroup$
    – R_hub
    Jul 4 at 10:41
  • $\begingroup$ I think I got your concerns, and this is really helpful thank you. There are only a few things that I do not understand yet. First, why does the treatment vary across races? It varies across states and across time, but is the same for all races within a state or am I wrong with that? Second, the specification would require to interact the $\deltaCOVID_{gst}^{I>60}$ with the year-month variable so that I can infer the monthly impact? Otherwise the coefficient just measures the overall average effect right? $\endgroup$
    – R_hub
    Jul 4 at 12:08
  • $\begingroup$ Third, what would be the interpretation of this set of interactions? Last, you added a subsript $g$, which would be the racial affiliation I guess? That means I run the regression for all non-white races, and $\delta COVID_{gst}^{I>60} measures the differential impact of high stringency index between non-whites and whites? $\endgroup$
    – R_hub
    Jul 4 at 12:19
  • $\begingroup$ I have a better handle on what you want to do. But isn’t each state reaching the threshold in different months? I think I can be more helpful once I figure out how you’re separating the pre- and post-treatment epochs. Unless you’re picking a month, say April, and looking at how many days each state was in the treated condition up until that point, then this could work. But is there a particular month for each policy which separates the before-and-after periods? $\endgroup$ Jul 4 at 20:33
  • $\begingroup$ Actually all States get the Treatment directly in the first Month (April 2020), as all States had implemented policies which result in a Stringency Index above 60. The treatment than continues with different intensities, but lets say if a State was exposed 20 days at a Stringency level above 60 in April 2020, then loosens policies and will be at 20 days exposure in May, while other States will have substantially higher days of exposure. In that sense the State is still treated, but at the same intensity as in the previous month, while for others it got more intense. $\endgroup$
    – R_hub
    Jul 5 at 7:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.