This I assume is an obvious question, but I usually hear it said that with difference in difference estimators, you don't need to worry about level differences between groups. I am wondering if this is true because- with difference in differences, we literally are not using across group changes to identify the effect? i.e. we are throwing out the across group variability, and only comparing the changes within groups with each other?
This I assume is an obvious question, but I usually hear it said that with difference in difference estimators, you don't need to worry about level differences between group.
In general, you do not have to concern yourself with level differences between treatment and control groups, but your analysis should address why this would not impact trends. Outcome levels may differ in the pre-period, but their time variation should be similar. We assume any differences in the pre- and post-periods would have moved in tandem in the absence of treatment. Thus, while initial differences across groups may exist, your slopes should not be trending differently over time before the onset of treatment. Difference-in-differences performs a 'double-difference' across groups and across times, and so the stable, time-constant differences in levels across treatment and control groups, which would include the level differences in your outcome pre-treatment, will be 'substracted out' using this approach.
I should note that "overlapping" trends is not a prerequisite. It is only necessary that you show—visually—a stable, inter-temporal evolution of the group trends prior to treatment exposure. A picture is worth a thousand words. Here is one reproduced from these slides:
Each experimental series is on a completely different level before the institution of the hypothetical treatment (i.e., vertical line). Yet, in spite of any level differences in the pre-period, each series is moving together as they approach the intervention. A good plot of the group averages over time affords you greater confidence that any post-treatment shift in trend is actually due to the intervention and not merely a consequence of one series proceeding on a completely different growth trajectory pre-treatment.
We literally are not using across group changes to identify the effect?
Identification stems from inter-temporal variation between groups. In essence, you are comparing the across-time change in your outcome for the treatment group with the across-time change in your outcome for the control group. In your post you indicate that this method is "throwing out across group variability" when identifying a treatment effect. I would argue that this method 'differences out' the stable, time-invariant level differences that exist between groups.