1
$\begingroup$

I am working on a project and I would like to try a difference-in-differences approach. I want to study how increases in network affect some outcome variable. Basically, I defined "treatment" to be 1 when an ID sees an increase in length compared to the previous period. The data looks something like this:

enter image description here

For ID 1, notice that treatment in 2003 is 1 since the network increased. In 2004, however, it is 0 because it did not increase.

I plan to estimate something like: $outcome = \beta_0 + \beta_1Year + \beta_2ID + \beta_3Treatment + e$

I interpret $\beta_3$ as the estimated effect of increasing the network.

I have never seen a difference-in-differences approach used with treatment defined in this manner, and I am having trouble finding examples in the social sciences literature. I would like to know if this is a valid approach, what the coefficient on $Treat$ signifies, and if you can point me to examples from the literature that use diff-in-diff in a way I have described here.

EDIT: It seems what I am looking for is something opposite of a "staggered" D-in-D approach, but I am not sure what that is.

$\endgroup$
4
  • $\begingroup$ Could you describe what a network increase is a bit more? Once the unit goes back to 0, is the unit really untreated in that period? $\endgroup$ Oct 7, 2021 at 16:58
  • $\begingroup$ @ThomasBilach If treatment moves back to 0, this just means the network is the same length as that in the previous observation. In other words, if the network increases from 2002 to 2003, treatment is 1, but from 2003 to 2004, if the network remains the same, treatment is 0. I define treatment as having an increase in the network compared with the previous year. $\endgroup$
    – z_11122
    Oct 7, 2021 at 17:01
  • $\begingroup$ @ThomasBilach I suppose the binary variable describes whether or not an ID received more intense treatment in period t compared to period t-1. Perhaps using a binary variable is not the correct approach $\endgroup$
    – z_11122
    Oct 7, 2021 at 17:05
  • $\begingroup$ Also, do you have a lot of missing data? Before a network increase, there are units missing upwards of 4 years worth of data. $\endgroup$ Oct 8, 2021 at 17:12

1 Answer 1

1
$\begingroup$

The treatment variable is allowed to switch 'on' and 'off' over time. This assumes, of course, that when the treatment dummy 'turns off' (i.e., switches from 1 back to 0), that the unit is, in fact, untreated. In your setting, however, I don't think you can justify defining treatment in this manner, as 'network changes' emerge incrementally over time. In other words, it appears your treatment increases in discrete jumps as time progresses. If my interpretation is accurate, then treatment isn't really reversing at all; treatment is actually moving up some intensity ladder over time, in which case the treatment variable should reflect this reality.

One approach involves defining your treatment variable in a different way. To begin, the treatment indicator should 'turn on' (i.e., switch from 0 to 1) at the first network increase and then 'stay on' until the next network changes occurs; it should actually never return back to 0, which would be consistent with the removal of treatment. Accordingly, by the next network change the treatment variable should take on a value of 2, then 3, then 4, and so on, depending upon how many network changes you observe over time. Please note, it doesn't appear untreated units start treatment at any one particular time period, so untreated units will remain consistently 0.

The model should include a full set of id effects, a full set of year effects, and the treatment variable. The basic R code would look something like the following:

lm(y ~ as.factor(id) + as.factor(year) + as.factor(treat), covariates, data = ...)

Note how I use as.factor(treat) in the model formula, which tells R to 'dummy out' each network change. In the manner in which I have defined treatment, the integer values denoting the network expansions (i.e., 1, 2, 3, 4, etc.) represent discrete jumps in intensity; it should not enter the model as a continuous variable. I'm not sure how many network changes units will undergo over time, but the model should return a treatment effect for each jump in treatment over time.

Issues/Concerns/Considerations

  1. Do any units experience a network change in the pre-treatment years? And, is any network change before the first shock significant enough to affect pre-treatment trends? In the abridged data frame shown above, some id's have distinct integer values for the network variable in the pre-period, while for other id's the network variable starts as 0 in the pre-period before the first increase occurs. How much movement is observed before treatment actually starts?

  2. This approach assumes homogenous treatment effects across time periods and across groups. For example, we assume the change in outcome is immediate—and constant—in the post-treatment years. Is this a reasonable assumption? Similarly, is the jump from 200 to 250 for id 1 the same as the jump from 100 to 400 for id 2? Both units experience their first "treatment" according to your definition, yet the changes vary so widely across different groups of id's. It might be worthwhile to run your analysis on subsets of id's experiencing similar network jumps. I am hesitant to recommend modeling network itself as a continuous treatment, but it could work, and it's as easy as replacing the binary measure(s) with the network variable as is. I don't know what a "network increase" really is in your setting, but modeling it as continuous isn't outside the realm of possibilities. The major concern I have is what constitutes the start of treatment in the continuous case? In other words, what distinguishes pre- versus post-treatment when the continuous variable itself defines the event(s)? Technically, a unit is considered "treated" whenever a network expands. For example, does id 1's network grow in any way before 2003, or is the network size/length relatively stable before this year? Is there a direct "network effect(s)" you're investigating? I suppose I would need more information about what "treatment" really is before I offer any further guidance.

  3. It seems you have a lot of missing data in the pre-period. Note how id 2 and id 9 are missing three consecutive years of data immediately before the first network expansion. Did you purposely omit these years? The continuity of the row numbers suggests real missingness, but I can't be sure.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.