1
$\begingroup$

This post simplifies how to conduct a pre-trend analysis using "coefficient plotting", which follows from lecture notes by Pischke 2005.

I simplified copying the description here:

A formal test which is also suitable for multivalued treatments or several groups is to interact the treatment variable with time dummies. Suppose you have 3 pre-treatment periods and 3 post-treatment periods, you would then regress

$$y_{it} = \lambda_i + \delta_t + \beta_{-2}D_{it} + \beta_{-1}D_{it} + \beta_1 D_{it} + \beta_2 D_{it} + \beta_3 D_{it} + \epsilon_{it}$$

where $y$ is the outcome for individual $i$ at time $t$, $\lambda$ and $\delta$ are individual and time fixed effects (this is a generalized way of writing down the diff-in-diff model which also allows for multiple treatments or treatments at different times). If the outcome trends between treatment and control group are the same, then $\beta_{-2}$ and $\beta_{-1}$ should be insignificant, i.e. the difference in differences is not significantly different between the two groups in the pre-treatment period.

So, it seems that one way to satisfy pre-trend testing is when the estimates of $\beta_{-2}$ and $\beta_{-1}$ are insignificant. But what about $\beta_j$, $j \geq 0$? From Pischke's notes, it seems that they should be identical but he did not mention whether they should be significant or not.

Apart from that, in situations where $\beta_{-2}$ and $\beta_{-1}$ are significant but close to zero, can it be counted as a parallel trend as Pischke (2005) as above?

$\endgroup$
1
+50
$\begingroup$

So, it seems that one way to satisfy pre-trend testing is when the estimates of $\beta_{−2}$ and $\beta_{−1}$ are insignificant.

Correct.

It's evidence supporting parallel trends in the periods before a treatment. If it fails in practice, then we often turn to synthetic control methods to help us select from a pool of eligible controls with comparable outcome trajectories pre-shock.

But what about $\beta_j$, $j \geq 0$?

What about them?

The estimates investigate whether treatment effects vary with time since exposure. For example, effects may accumulate or deteriorate with the passage of time. The lecture notes you cite make no claim about the significance of the $\beta_j$'s; the only assertion is that the effects in the post-period may not be identical. It depends entirely on how the intervention (i.e., treatment) affects your outcome of interest. It's not a requirement that the $j$ lags have any particular direction or magnitude. Sometimes a discovery that all of the $\beta_j$'s are zero may be an interesting finding as well. Not every evaluation returns a significant treatment effect.

Apart from that, in situations where $\beta_{−2}$ and $\beta_{−1}$ are significant but close to zero, can it be counted as a parallel trend as Pischke (2005) as above?

It's hard to say without knowing more about your treatment.

It could be interpreted as selection bias. It doesn't necessarily mean you have to abandon your project, but you should ask yourself why you're observing a difference in trend in the pre-period. Moreover, what's the direction of the bias? Do the effects grow stronger as you approach the immediate adoption period? Was the policy uniform/staggered in its roll out? Does theory suggest units anticipated the policy? My answer would depend heavily on how you answered the following questions.

The short answer is, maybe. It's less of a concern when there's a strong theoretical foundation suggesting an anticipatory response to treatment.

$\endgroup$
6
  • $\begingroup$ Thank you @Thomas Bilach. I have a couple of questions as below : (1) "If it fails in practice, then we often turn to synthetic control methods to help us select from a pool of eligible controls with comparable outcome trajectories pre-shock" I did not understand what does this mean here. Can you explain it to me intuitively? Thanks a heap $\endgroup$
    – Louise
    Jul 7 at 8:17
  • $\begingroup$ (2)I thought "$\beta_{−2}$ close to 0 but significant" will be analogous to "$\beta_{−2}$ not close to 0" but insignificant. But from your explanation, it is not the case. So, "$\beta_{−2}$ close to 0 but significant" can be counted as anticipatory effects? I am surprised about that because the immediate difference between treatment and control group close by 0, so how come the anticipatory effect happen. $\endgroup$
    – Louise
    Jul 7 at 8:25
  • $\begingroup$ (3) Could you please explain it clearer to me, I did not get this idea fully "Was the policy uniform/staggered in its roll out?" $\endgroup$
    – Louise
    Jul 7 at 8:32
  • $\begingroup$ (1) The time-varying effects, in particular the coefficient leads, trace out deviations from the common trends that your units experience in the periods approaching the shock (i.e., treatment). If you do detect pre-trends, then you should look into why this is occurring, or try working with a new subset of units with similar pre-period trends. $\endgroup$ Jul 8 at 21:01
  • $\begingroup$ (2) I'm not making this claim. You asked me whether or not a lead that is bounded away from zero, but pretty close to it, is evidence of pre-trends. Technically, it is evidence of a deviation from a common trend, but it may not be much of a concern if you suspect anticipatory behavior. So, for example, it's reasonable to assume firms change their behavior in response to impending regulation. It's not so much that they were trending apart, but rather that they were anticipating a policy change! $\endgroup$ Jul 8 at 21:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.