Interpreting significance in difference in difference results I have a panel dataset of two groups X1 (control) and X2 (treatment), and my metric of interest Y (for both control and treatment). I have a dummy variable indicating what group they belong to, and a dummy variable indicating whether it is before or after intervention.
If I do a classic Difference in Difference analysis (Y ~ time+ treatment + time * treatment) it shows time (pre/post) as significant, but neither treatment or time * treatment are significant.
However, if I first calculate the difference between X2 and X1 (diff = X2-X1), and regress time against diff (diff ~ time) time is significant, and time has the exact same coefficient as time * treatment (since its the mean difference).
How should I interpret these results/differences in significance? 
Do I interpret it as: while the absolute difference between the control and treatment between the two periods is significant, the impact of the difference between the two periods outweighs that of the difference between the groups?
 A: If you have measurements before and after some treatment, you have repeated measures. These are correlated to one another and thus violate the assumption of independent measurements. This means that neither of the models you have run produce valid standard errors.
Using a linear mixed effects model, you can account for the dependence between measurements of the same experimental units. For example, in R's lme4:
library(lme4)
LMM <- lmer(y ~ time * treatment + (1 | ID))
summary(LMM)

ID would then have to be a variable that indicates the experimental unit.
(Assuming the conditional distribution of your outcome is indeed normal.)

Apart from this, there is another difference in the first and second model you have run: The second model has a single error for every pair of $\{X_1, X_2\}$. This would only be valid if you could meaningfully construct such pairs (i.e. the treatments and controls are the same experimental units). Even if that were the case though, it would be more powerful to use a mixed model.
