2
$\begingroup$

I have a data set in which I have treatment, event, and treatment*event vectors, along with other variable vectors that I have computed.

I ran my regressions and find my coefficients and their significance.

Now, I want to run a robustness check using propensity score matching (PSM). I don't know much about it, but from what I learned, PSM is a way to run difference-in-differences by considering only comparable treatment and control observations, eliminating the unmatched observations. Then, if the treatment*event coefficient is consistent with the one previously estimated, then my results are robust. RIGHT?

I DON'T UNDERSTAND: I saw videos in which to find propensity scores they run a regression putting the treatment vector as the dependent variable. Shouldn't it be the treatment*event vector instead?

Then, I see that the analysis gives the difference between the means of the 2 groups and their significance. Is that estimated difference my new treatement*event coefficient that must be compared with the old one to see if it is consistent?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

There are two ways to use propensity score for diff-in-diff. One way is to simply make the control group (i.e., the group that will not be treated in the post-period) resemble the treated group (i.e., the group that will be treated in the post-period). In this case, treatment group membership is the dependent variable in the propensity score regression model: there are only two groups.

The other way is described in Stuart et al. (2014). This involves using propensity scores to make the four "groups" (control-pre, treated-pre, control-post, treated-post) all resemble each other. This is particularly useful when the people you measured in the post-period are not the same as the people you measured in the pre-period, such as when studying the effect of a policy using annually collected random samples. This seems to be the alternative method you are suggesting.

When the same individuals are measured at both time points, the pre units already resemble the post units, because they are the same units! So there is no need to do work to ensure the pre- and post-period groups resemble each other using propensity scores. It is sufficient to ensure the pre-period groups resemble each other because the post-period groups will resemble each other and the pre-period groups automatically (again, because they are the same units).

After matching, you will run your diff-in-diff analysis just the same as you would have without matching; the treatment*event coefficient is still the treatment effect.

You can also just compare the post-period means in the matched sample instead of using diff-in-diff; this requires a different set of assumptions than diff-in-diff. Namely, the difference in matched means requires the assumption of strong ignorability to be unbiased, whereas diff-in-diff requires the conditional parallel slopes assumptions and the assumption of no treatment dynamics (i.e., pre-period outcomes do not affect treatment status). So, you can use matching as a way to strengthen diff-in-diff, or you can use it as a separate method that relies on different assumptions. From your description, it's not immediately clear what the videos you watched did or why.

Improve this answer
$\endgroup$
8
  • $\begingroup$ Hi @noah, thanks for answering. My data is a set of stocks which at a given point split into two groups: one was subject to the introduction of a policy on financial markets, the other was not. My study aims to understand whether this policy has affected the trading of the treated stocks by looking at different daily factors (spread, volatility, volumes ecc). I run a diff-in-diff regression for each of those factors. I wanted to use PSM to match stocks with similar market capitalization (which is monthly across my data set, while all the other factors that I am analyzing are daily). $\endgroup$
    – Mining
    Mar 2 at 11:34
  • $\begingroup$ So i think I should use the first method that you mentioned, given that the pre and post subjects are the same (even though the market cap has changed over time). But I don't understand one thing: once I have run my PSM and obtained my estimate of the difference in mean between the 2 groups, how can I run the regression again? what should be the dependent variable? $\endgroup$
    – Mining
    Mar 2 at 11:37
  • 1
    $\begingroup$ @ThomasBilach If you want to rely on strong ignorability conditional on the background covariates and pre-intervention outcomes, then you should just perform a simple post-period difference in means. But if you want to rely on conditional parallel trends, then I'm not sure if you should include the pre-intervention outcomes in the matching, but you should do a diff-in-diff in the matched sample. I recommend this paper that goes more into it from the perspective of regression: osf.io/5zdme $\endgroup$
    – Noah
    Mar 9 at 6:24
  • 1
    $\begingroup$ If the distributions of the pre-treatment outcomes in treatment & control groups are significantly different, does Daw & Hatfield's 2018 paper rule out using propensity score matching in diff-in-diff because of regression-to-the-mean effects? onlinelibrary.wiley.com/doi/abs/10.1111/1475-6773.12993 $\endgroup$
    – RobertF
    Apr 26 at 14:12
  • 1
    $\begingroup$ @RobertF yes, that is a critical paper that everyone should read! If using diff-in-diff, one should only match on time-fixed covariates. This is why it is bad to match on the pre-treatment outcomes if using diff-in-diff. If using difference in means, though, you should match on the pre-treatment outcomes, but this analysis relies on different assumptions. $\endgroup$
    – Noah
    Apr 26 at 15:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.