2
$\begingroup$

This might be a bit basic, but I'm struggling to determine the ideal level of aggregation of the data for an Event Study analysis.

I wish to study if a specific policy affected fertility in certain groups, and have microdata at the mother level (age, ethnicity, education, state, and year of birth of each child).

The problem is that I don't know what level of aggregation to choose. In other words, I could collapse the data such that I have the number of births by state and year (only a few states were affected by the policy), or I could do it in a way that I would have more info, such as the number of birth in each state by mother's age and education group.

I think what I mean is what to collapse by, i.e get the sum of births by state and year as opposed to by state, year, age, ethnicity, and education groups. That is, the unit of observation should be only the state-year (treatment vs control states) or should I add more information in a way that it's state-year-education-age.

Once I have figured out the level of aggregation, I will perform a Differences-in-Differences (TWFE to be more precise) analysis in the form of an Event Study, and if I go with the more disaggregated data, will include fixed effects for each grouping variable (education, mother's age, etc.).

I very much appreciate your help and thank you in advance.

$\endgroup$
6
  • $\begingroup$ Can you tell us a little more about the policy? Was it intended to impact a particular group? Or maybe it affects everyone and you want to follow particular cohorts over time? $\endgroup$ Commented Apr 5, 2021 at 16:07
  • $\begingroup$ Hi Thomas, it was a policy affecting some specific states, but probably differently across demographic groups. More precisely, some states made contraceptive methods free for poor women, each state at a different time. Using my data, I can see that such women are likely to be young, black/mixed race and less educated. $\endgroup$
    – Silas B.
    Commented Apr 5, 2021 at 17:12
  • 2
    $\begingroup$ Is the treatment down at this lower level defined by you? Or was this a policy specially designed for women below a specific income threshold? This study may be a candidate for difference-in-difference-in-differences. $\endgroup$ Commented Apr 5, 2021 at 17:28
  • $\begingroup$ It was defined by me. I'm planning to run a triple differences estimation as well, but I figured I should first start with a simple double difference first... $\endgroup$
    – Silas B.
    Commented Apr 5, 2021 at 18:24
  • 1
    $\begingroup$ In my view there's a trade-off: if you can guarantee parallel trends conditioning on less variables, you may be more likely to find an effect because of increased variability. If not, then adding covariates is the way to go, as the trends may be parallel conditioning on more variables (but you'll have less variability in Y). I can't say much about collapsing the data on more variables though. $\endgroup$
    – Caio C.
    Commented Apr 5, 2021 at 18:28

1 Answer 1

0
$\begingroup$

The policy is introduced at the state level, affecting some states but not others. I think you’re interested in plotting your coefficients, which is typical in event study frameworks. It is convenient to use a regression formulation as it allows you to include covariates. Sometimes we can ensure parallel trends only after conditioning on observables.

Your question indicates you’re dealing with a policy introduced at the state level, but you also suspect a subgroup at the $i$-level is more sensitive to the policy. First, let's specify a difference-in-differences (DD) model with a state level policy variable. Since the policy isn't uniform with respect to its introduction, you must proceed with the 'generalized' DD estimator. The model would look something like the following:

$$ y_{ist} = \gamma_s + \lambda_t + \beta DD_{st} + X_{ist}\delta + \epsilon_{ist} $$

where $y_{ist}$ is the fertility rate for mother $i$ in state $s$ in year $t$. The parameters $\gamma_s$ and $\lambda_t$ denote fixed effects for states and years, respectively. The variable $DD_{st}$ is a policy dummy equal to 1 if state $s$ adopts the fertility policy and is in the $t$ periods after treatment. $X_{ist}$ denotes a vector of time-varying covariates at the state and individual level. In addition to state level covariates (e.g., unemployment rate), $X_{ist}$ may also include age, education, number of intimate partners, et cetera, for each $i$. If you can ensure parallel paths conditioning on time-varying observables, then by all means include them. Note the estimate of $\beta$ is a static treatment effect. It does not investigate the dynamics of exposure.

A quick word with respect to aggregation. Only state level regressors that vary over time matter with respect to the identification of your treatment effect. Thus, your estimate of $\beta$ should not vary if you estimated the following:

$$ y_{st} = \gamma_s + \lambda_t + \beta DD_{st} + X_{st}\delta + \epsilon_{st} $$

where now the data is aggregated up to the state level. Note how $X_{st}$ contains only the state-year regressors. It is perfectly valid to estimate a DD equation using the state-year observations. But in my opinion, I would estimate $\beta$ using the micro data. Although the individual level covariates don't offer much value in the identification department, they may help lower your standard errors.

You can still plot the evolution of the DD coefficients over time using your micro data. One approach to take is the following:

$$ y_{ist} = \gamma_s + \lambda_t + \sum_{\tau = -K}^{-2}\beta_{-\tau} DD_{s,t-\tau} + \sum_{\tau = 0}^{L} \beta_{\tau} DD_{s,t+\tau} + X_{ist}\delta + \epsilon_{ist}, $$

where each treated jurisdiction has its own event window around the initial year of adoption. In practice, the time to treatment is expressed as a series of policy indicators for periods relative to the adoption year: $\tau = -K,..., -4, -3, -2, 0, +1, +2, +3, +4,..., L$. The first adoption year is at $\tau = 0$. This is often viewed as the immediate effect of the policy. The policy indicators where $\tau < 0$ usually denote treatment leads, while the indicators for $\tau > 0$ typically denote treatment lags. It is also quite common in applied work to let $DD_{s,t<0}$ denote any arbitrary number of lags. It doesn't matter if the lead-lag notation was reversed—just make sure it's consistent. The goal is the plot the $\hat{\beta}_j$'s over time.

If you were to saturate the model with policy indicators for all periods before and after treatment, it is customary to drop the period before policy adoption (i.e., $-1$). But it doesn't have to work this way. It is also quite common to drop a later period (e.g., $\tau = -5$). Estimates should be interpreted as relative to the excluded year. For example, $\tau=-3$ is interpreted, in general, as the average mean difference in outcomes for states 3 periods before the event. The counterfactual at this relative period is the cohort of non-adopter states (i.e., the always $0$'s) and those that are soon-to-be-treated. Even though some states may be treated later, they serve as a counterfactual for early-adopters.

A saturated parameterization is not a requirement. If the exposure is transient, then you may want to report only a finite number of leads and/or lags. Suppose New York and California adopt a new law. New York adopts the legislation in 2018, while California lags, officially rolling out the policy in 2019. Now suppose you want to investigate anticipatory effects. Individuals may anticipate impending legislation and change their behavior. A policy "lead" variable is a dummy equal to 1 for New York in 2017 and 1 for California in 2018, 0 otherwise. Note how it equals 1 for states in the treatment group and in the year before the policy is implemented. This is very similar to a standard interaction term, but we must instantiate the variable manually to account for states with different start times. Similarly, the immediate effect of the policy is a dummy equal to 1 for New York in 2018 and 1 for California in 2019. Repeat this process if you want to trace out the full dynamics of exposure.

A triple difference lends more credibility to the impact of the policy. I suppose mothers below a specific income threshold is another dimension along which the treatment may vary. Let $p$ equal 1 for a mother living in poverty. A difference-in-difference-in-differences (DDD) model would look something like the following:

$$ y_{ipst} = \gamma_{st} + \lambda_{pt} + \theta_{ps} + \beta DDD_{pst} + X_{ipst}\delta + \epsilon_{ipst} $$

where the policy now varies along three dimensions: state $s$, year $t$, and poverty status $p$. Mothers in group $p$ may be more sensitive to the policy, especially if the new legislation was specifically designed to support poor mothers. Here, regress $y_{ipst}$ on state-year interactions, poverty-year interactions, poverty-state interactions, and the policy dummy. This formulation may seem unnatural, but the triple interaction term is implicit in the coding of the policy variable. $DDD_{pst}$ is equal to 1 if the state adopts, and a mother is below a specific income threshold, and it is a post-period, 0 otherwise. We must proceed in this manner as the timing of the new legislation isn't standardized.

I don't know what measure you're using to split mothers into different income groups, but using their pre-tax cash income in a period before the intervention is a safe choice. I would also include a rich set of covariate adjustments at the individual level to ensure balance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.