I have a panel dataset of two groups X1 (control) and X2 (treatment), and my metric of interest Y (for both control and treatment). I have a dummy variable indicating what group they belong to, and a dummy variable indicating whether it is before or after intervention.
If I do a classic Difference in Difference analysis (Y ~ time+ treatment + time * treatment) it shows time (pre/post) as significant, but neither treatment or time * treatment are significant.
However, if I first calculate the difference between X2 and X1 (diff = X2-X1), and regress time against diff (diff ~ time) time is significant, and time has the exact same coefficient as time * treatment (since its the mean difference).
How should I interpret these results/differences in significance?
Do I interpret it as: while the absolute difference between the control and treatment between the two periods is significant, the impact of the difference between the two periods outweighs that of the difference between the groups?