1
$\begingroup$

I am currently working on an empirical design that, at first sight, reminds me of a classical difference-in-difference (DiD) setup. Let me give you an (imaginary) example: Assume we are observing bank loans that were originated between 2010 and 2015 and how they are traded (priced) on the secondary market, meaning that we collect weekly pricing data for all loans that were originated in that time span. Now, in 12/2013 there was a policy intervention. This policy intervention changed the structure of all loans that were originated thereafter. However, with the policy intervention, some loans originated after 12/2013 were affected even more if they fulfilled some specific loan features.

Building on a two-way fixed effects model, this can be written down as:

$y=a_i + a_t + beta^{dd}D_{it} + e_{it}$

Again, this reminds me of a typical DiD setup. However, the problem is that in my case, the data is not available as in a typical DiD setting. Usually, you observe the treatment and control group post- and pre-treatment.

However, in my example, only the loans that were originated after 12/2013 can possibly be treated. The loans that were originated before 12/2013 can never become treated, because they are on existing contracts and cannot be changed by law. This means, I do not observe the treated loans in the pre-treatment period because all the treated loans were by design originated after 12/2013.

What I find confusing is whether this design is still some form of DiD, or is it just basic pooled OLS? Because we have observations in the pre- and post-period, and we have observations for treated and untreated units. However, what we do lack though, is observing the units that were later treated even before the intervention. And this means that there is no way to check whether parallel trends hold, right?

Looking forward to some mastermind in empirical identification to enlighten me. Literature and other references on this topic would be very much appreciated.

Edit: A thought came into my mind, which I want to also bring into the discussion. Could the exemplary design above be somewhat similar to an DiD design with different treatment intensity? Because all of the loans originated after 12/2013 are affected by the regulation, however, loans that have the specific feature are affected with a different intensity. Are the loans that originated pre 12/2013, for which we still have pricing data, a valid control group?

$\endgroup$
3
  • $\begingroup$ Unless I misunderstood, aren’t you following the pricing of these loans before and after the intervention? I assume the confusion is because all units are treated, but I could be wrong. Also, was the policy specifically directed at loans with specific features? $\endgroup$ Commented Oct 13, 2022 at 14:25
  • $\begingroup$ Yes, I am following the pricing of the loans before and after the intervention. However, "treated" loans were all originated after the treatment. The treatment did not affect loans originated pre treatment because the existing contracts cannot be changed. The policy was directed at all loans, so it affected all loans originated after the intervention. However, some loans, the ones with specific features, were even more impacted by the intervention. $\endgroup$
    – n_arch
    Commented Oct 13, 2022 at 21:20
  • $\begingroup$ So in essence, the treatment group consists out of loans that were originated post treatment. We do not observe pricing for those loans before the intervention, as they were not originated back then. I am unsure whether this is some form of DiD setting. Because in a typical DiD setting you of course observe the Treatment group after AND before the intervention.. Any idea? Or something where I can find something similar to read? $\endgroup$
    – n_arch
    Commented Oct 13, 2022 at 21:22

1 Answer 1

0
$\begingroup$

Now, in 12/2013 there was a policy intervention. This policy intervention changed the structure of all loans that were originated thereafter.

This is problematic.

The "treated" loans originated as a direct result of the policy. Those loans have no pricing data before the policy went into effect. In short, you won't have any reliable way to inspect the inter-temporal evolution of the pricing trends between treated and untreated loans before the policy.

However, with the policy intervention, some loans originated after 12/2013 were affected even more if they fulfilled some specific loan features.

This is good news.

You now have a source of variation to exploit. But it appears to me that you're outside the realm of the "classical" difference-in-differences setting. Pre-policy data is a requirement.

Usually, you observe the treatment and control group post- and pre-treatment.

Correct.

We should be observing the pricing of loans in the months before the policy. Unless my understanding is incorrect, the policy resulted in new types of loans, none of which you observe before the policy. The pre-policy loans were not, or should I say cannot, be eligible for this policy.

What I find confusing is whether this design is still some form of DiD, or is it just basic pooled OLS?

You may have to resort to a posttest-only evaluation of loan pricing.

...[T]his means that there is no way to check whether parallel trends hold, right?

Correct.

Could the exemplary design above be somewhat similar to an DiD design with different treatment intensity?

It's more or less similar, but without any pre-event data.

All of the variation is between loans with different features in the post-policy period.

Are the loans that originated pre 12/2013, for which we still have pricing data, a valid control group?

Not in my estimation.

Answer this: do you believe the pre-policy trends approximate what would have happened in the "treated loan group" had no policy been introduced? This doesn't seem valid to me. The other issue is your policy affects the pricing of all loans after December of 2013. Thus, we don't observe the pre-/post-policy evolution of pricing trends for a clean group of "untreated" loans. This means your counterfactual is the pricing trajectory of loans in the pre-policy era, a group which had no eligibility for the policy anyway.

In short, I don't think a traditional differences-in-differences analysis applies here. But don't lose hope. Focus on the variation you do have over time, which for now is the pricing trends across loans with different features.

$\endgroup$
6
  • $\begingroup$ Thank you for providing your comprehensive answer. I already thought about exploiting the variation in the reaction of the loans (depending on whether they posess the given feature or not). However, I am unsure what kind of assumptions go with this. It won't be the classical DiD assumptions as we rely only on post-intervention variation then. $\endgroup$
    – n_arch
    Commented Oct 17, 2022 at 7:41
  • $\begingroup$ And I think you are mostly correct with "the policy resulted in new types of loans, none of which you observe before the policy". Indeed, ALL loans that were originated after the policy needed to be structured differently. $\endgroup$
    – n_arch
    Commented Oct 17, 2022 at 8:37
  • $\begingroup$ No problem. In regard to your second comment, is my statement somewhat incorrect? According to your post, these loans did not exist before December. Right? $\endgroup$ Commented Oct 17, 2022 at 14:17
  • $\begingroup$ No, your statement is correct, I just wanted to make sure that it is clear that after 12/2013 not just the structure of some loans but the structure of literally all loans that were originated thereafter has changed. $\endgroup$
    – n_arch
    Commented Oct 17, 2022 at 14:51
  • $\begingroup$ You suggest to "resort to a posttest-only evaluation of loan pricing." However, this would result in the usual endogeneity concerns like "The loans (with and without the specific feature) were not different in the first place". On the other hand, the design as it is right now doesn't really circumvent this problem because we cannot controll for the difference between treated and untreated in pre-event period as we cannot observe this, am I right? $\endgroup$
    – n_arch
    Commented Oct 17, 2022 at 15:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.