1
$\begingroup$

I am trying to analyze the effect of receiving "elite" status on a university's number of international first-year students. I have a perfectly balanced panel of 17 universities over an 18-year time period. It consists of universities who received and retained the elite status, some universities who received and lost the elite status, and some who made it to the final round, but were eventually not awarded the elite status. The universities received the elite status at 3 different points in time.

My understanding is that DiD allows for the treatment to be "switched off" again (although much of the literature seems to discuss it in terms of "before" and "after," suggesting that the treatment will remain in place forever after it is "switched on").

Can you suggest any literature that analyzes a similar situation (where the treatment is "switched off" again), or anything else that might be important here, also for DiD? I know the literature regarding standard errors following the Bertrand et al. (2002) paper and wild bootstrap.

$\endgroup$
2
  • $\begingroup$ Welcome. There is no requirement that a treatment be “permanent” in the sense that you describe. Do you want to know which studies analyze such treatments, or the “fixes” for it in practice? Sometimes the estimates tend to misbehave when there is staggered entry or exit, or when the treatments switch on and off repeatedly over time. $\endgroup$ Commented Feb 2 at 6:02
  • $\begingroup$ @ThomasBilach I am not sure if OP is interested in the studies you allude to (although that is my reading) -- where there is staggered entry and exit, and particularly where there is repeated switching into and out of treatment -- but I certainly would be. Please cite. $\endgroup$
    – Relevance
    Commented Feb 3 at 6:29

1 Answer 1

0
$\begingroup$

In a separate answer authored here, I note at least three studies where the pattern of a binary policy variable goes back and forth between 0 and 1 multiple times over time. The time period where the policy variable switches from 1 back to 0 is the period where the policy has been revoked. In the world of policy analysis, a treatment is rarely permanent. And even when a policy is never revoked or withdrawn, it may be reasonable to assume the treatment no longer affects the outcome of interest after some time has passed. In such a setting we may even force the policy variable to "turn off" after a while (i.e., switch from 1 back to 0). In other settings, the parameters of a policy's sunrise and sunset date are clearly defined. For the treated unit(s) where this information is known, we should define the policy variable in such a way that we allow for the "on" and "off" periods of the policy. There is no requirement in a difference-in-differences setting that a policy remain in effect in every single time period.

My understanding is that DiD allows for the treatment to be "switched off" again (although much of the literature seems to discuss it in terms of "before" and "after," suggesting that the treatment will remain in place forever after it is "switched on").

Correct.

The terminology "before versus after" or "pre versus post" does not imply a permanent treatment, though it sometimes does in practical applications. A treatment is allowed to "turn off" (i.e., switch from 1 back to 0) for some, or all, of the treated units.

There isn't really a consensus across disciplines regarding how we should describe the pre- and post-treatment epochs in scientific writing, especially when we have multiple pre- and post-periods and where the lengths of the exposure periods vary in between treatments. For example, the term "post-treatment" is one of the most common phrases used to describe the epoch when a policy is in effect. Note how the prefix "post-" as a modifier means "subsequent to" the treatment. In this sense, it is often used synonymously with the word "after" treatment. In essence, we are referring to the periods "after" the treatment's start date, not necessarily after the policy is no longer active. But now suppose the policy has a specific date when it ends. Using the phrase "post-treatment" to describe unit-time combinations where no policy is in effect may confuse readers. I'd recommend using phrases like "active" and "inactive" to describe the "on" and "off" periods, respectively. Just be very detailed about the period(s) where treatment "reverses" and units have returned to the control condition. The more detailed, the better. Sometimes a good treatment variation plot removes any and all confusion. Please note that my answer assumes the start (end) dates vary, so the binary policy variable must be coded in a specific way. Review my answer above for a detailed introduction.

In your particular setting, I don't imagine the conferral of "elite" status occurs for all treated universities in the same year, so you must proceed with the generalized difference-in-differences estimator. This estimator allows your policy variable to take on almost any pattern, thought we should be clear about this pattern. From what I've gathered in your post, some achieve elite status early, others later. Some have their status taken away, others retain it. There appears to be a staggered, stepwise treatment assignment process, whereby elite status affects different institutions at different times. Not only that, but some universities achieve it and lose it, with this process never repeating. This suggests that your dichotomous treatment variable switches off for some universities—but never back on again. We still code the treatment variable in the same way. To be precise, the treatment variable equals 1 in the years when a treated university attains this status, 0 otherwise. That "0 otherwise" also applies to the university-year combinations when elite status is rescinded. Universities that almost receive elite status but unfortunately never make the cut remain consistently equal to 0.

You might be wondering why we should care about the "after" (i.e., "off") periods. Well, when assessing the effect of elite status on the number of international matriculants, my recommendation is to view the "revocation" of this lofty status as another type of treatment. Assuming international students have a particular attraction to exclusive universities, it would be interesting to observe enrollment percentages decline once universities lose their status. You may want to consider modeling the "off" periods directly to see whether this effect persists, if at all.

I'd recommend the etwfe package authored by Jeffrey Wooldridge in light of the uncommon (i.e., staggered) entry of universities into treatment. Review this vignette for a very gentle introduction. Wooldridge 2023 has even expanded upon his work to allow for full flexibility. It relies on throwing a sort of kitchen sink of interactions into the model. Read section 7.2 for the strategies when faced with reversible treatments (i.e., switching 'on' then 'off' again without a repeated bouncing back and forth between 0 and 1).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.