Assume we are doing a randomized experiment. The dependent variable is $Y$. Usually, we randomly put some of the subjects in the treatment group ($n$ subjects) and the rest to the control group ($m$ subject). Then, before the treatment, we measure $Y_i$ for $i\in T$ (treatment) and $Y_j$ for $j\in C$ (control). Then we give the treatment group the real treatment and give the control group placebo. Then after some time, we measure the $Y'_i$ for $i\in T$ (treatment) and $Y'_j$ for $j\in C$ (control).

Now we consider two ways of measuring the treatment effect.

  1. directly take the difference between the 2 groups after the treatment $\sum_{i=1}^nY'_i/n-\sum_{j=1}^mY'_j/m$

  2. take the difference in difference $\sum_{i=1}^n(Y'_i-Y_i)/n-\sum_{j=1}^m(Y'_j-Y_j)/m=\sum_{i=1}^nY'_i/n-\sum_{j=1}^mY'_j/m-(\sum_{i=1}^nY_i/n-\sum_{j=1}^mY_j/m)$

If the randomization is perfect and if $n$ and $m$ are very large, then we should have $\sum_{i=1}^nY_i/n-\sum_{j=1}^mY_j/m=0$, which makes the 2 methods the same. However,if $n$ and $m$ are not very large, then $\sum_{i=1}^nY_i/n-\sum_{j=1}^mY_j/m\neq 0$, then which method should I use? Is there a superiority of one over the other?


1 Answer 1


Assume your dependent variable $Y$ is continue. Then

1: $Pr(\sum_{i=1}^nY_i/n-\sum_{j=1}^mY_j/m=0) = 0$

2: $Var(\sum_{i=1}^nY'_i/n-\sum_{j=1}^mY'_j/m) \ge Var(\sum_{i=1}^n(Y'_i-Y_i)/n-\sum_{j=1}^m(Y'_j-Y_j)/m))$ given $Cov(Y_i,Y'_i) > 0.5Var(Y_i)Var(Y'_i)$, and it is very common.

  1. So should use DID in estimate and test. The test can be carried out by performing the grouped (trt vs control) t-test on the individual difference ($Y_i-Y'_i$).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.