Propensity Score Matching in a Generalized Difference in Difference

Question

Let's say that I have a generalized difference in difference specification $y_{i,t}=\gamma_i+\eta_t+\beta treat_i\times post_{t,i}+\epsilon_{i,t}$, where $i$ is a panel of countries. In this regression, the shock that the $post$ variable is proxying for happens at different times in different countries, so it depends on $i$. Someone told me that, since the countries I am looking at have different characteristics, that I should do a propensity score matching based on certain characteristics on the countries in the immediate pre-treatment time period. I understand intuitively what the person is going for, but I am a bit confused about how to do this test mechanically.

My understanding of propensity score matching is that I would be running a regression that looks like $y_{i,t}=\gamma_i+\eta_t+\beta treat_i\times post_{t,i}+\psi p(X_i)+\epsilon_{i,t}$, where the $p(X_i)$ is the probability that country $i$ gets assigned to treatment based on covariates observed in the first year prior to the treatment occurring in country $i$, represented here by the vector $X_i$. However, the $\gamma_i$ fixed effects should soak this propensity score control variable, so the treatment effect should be the same. I am wondering how my specification above is incorrect. I have never run a propensity score matching specification in practice, so I know that my issue is probably with lack of true understanding about how this process works. Pointing me towards any external sources on this question would also be welcome.

Answer 1

The variable $Post_{t}$ is typical notation used in the classical case. In your setting, however, the post-periods vary across countries, hence why it is $i$- and $t$-subscripted. I am partial to dummy variable notation, where the interaction term is implicit in the coding of the treatment dummy:

$$ y_{it} = \gamma_{i} + \eta_{t} + \beta D_{it} + \epsilon_{it}, $$

where $D_{it} = treat_{i} \times post_{it}$. The variable equals 1 for treated countries and in all periods where the policy is in effect, 0 otherwise.

I hope that when you say "shock" you are, in fact, referring to your treatment. If it's referring to macro-shocks that affect treated jurisdictions differently than untreated jurisdictions, then this is problematic. However I presume, given the circumstances, you're referring to the onset of a treatment than impacts different countries at different times. If so, then your model is correct.

In general, the advice you were given is correct, but it isn't comprehensive. For instance, the country fixed effects adjust for all time-constant factors that vary across countries. In the pre-treatment phase, for example, countries may differ widely in terms of their land area and various other topographical features. Some of these country-specific characteristics may even influence their cultural practices or even their style of governance. If any of these factors do change, they may change slowly. If they do not exhibit any variation over the course of the observation period, then your model considers them fixed. The stable differences across countries is already adjusted for by the country fixed effects.

Selection of countries based upon time-invariant criteria is allowable. In other words, a level imbalance across $i$ may drive the selection process. Identification is threatened whenever the probability of selection is varying over time, such that eligibility is increasing/decreasing as treatment nears. This complicates inference as a differential outcome trajectory may have already been emerging pre-shock.

The model you're proposing is attempting to adjust for inequalities in the covariate vector in a pre-period; note how all the variation in this vector is across countries. Again, the fixed effects already adjust for baseline level imbalances. If you're concerned about covariate imbalance, then simply include the observables on the right-hand side of your equation. In general, the covariates should affect the treatment and your outcome, but they should not be outcomes of the treatment itself.

Now to be clear, only time-varying covariates at the country level will likely be a source of omitted variable bias. This assumes your shock is, in fact, a country level treatment. The goal is to ensure a stable evolution of the group trends over time. I recommend including time-varying covariates (i.e., $X_{it}$) at the country level if they help ensure parallel trends.

Lastly, I recommend matching on pre-intervention outcome trends, though this approach has its downsides, especially in settings with small $T$. In particular, the matching estimator cannot fully discriminate between transitory fluctuations due to the error term and actual persistent, structural changes in outcome trends. If matching is necessary to ensure trend equivalence in the pre-treatment epoch, then I recommend juxtaposing both the unadjusted and propensity-score adjusted difference-in-differences estimates.

This post may address any remaining concerns you have regarding using propensity scores to estimate a policy change.