4
$\begingroup$

Is there a specific rule to the number of units in the treatment and control group. I am trying to carry out a difference-in-difference estimation and in the treatment group I have about 9000 individuals and in the control group I have about 800 individuals.

$\endgroup$
3
$\begingroup$

There is no direct rule but it will be more difficult to find a treatment effect if the two groups are very different in size. For example, if your outcome $Y$ is a normal random variable then the minimum detectable effect for the average treatment effect is $$\sqrt{\frac{\text{Var}(\widehat{Y})}{n}}\sqrt{\frac{1}{p(1-p)}}(q_{1-\frac{\alpha}{2}}+q_\lambda)$$ where $p$ is the proportion of individuals in the treatment group, $n$ is the sample size, $q$ are the quantiles from a standard normal, and $\alpha$ and $\lambda$ are the level and power that are chosen by the econometrician. The smallest minimum detectable effect is achieved if the number of treated individuals is equal to the number of control group individuals, i.e. $p=0.5$.

If you have very large differences in the group sizes then the treatment effect must be stronger in absolute value in order to be detected by your significance tests. For more information on minimum detectable effects see here.

$\endgroup$
  • $\begingroup$ Are you suggesting I reduce the sample size of the treatment group to a lesser amount or equal to the size of the control group. $\endgroup$ – Judith Aug 11 '14 at 13:48
  • $\begingroup$ The first best would be to increase the number of individuals in the control group but I acknowledge that this isn't often possible. Deleting individuals from the treatment group reduces the sample size, so the minimum detectable effect increases again. Whether this buys you something or not depends on the variation in the outcome. As I said: there is no rule for how many units you should have in either group but you should be aware of the potential limitations it can have. $\endgroup$ – Andy Aug 11 '14 at 13:52
  • $\begingroup$ @Andy do you have by any chance more reference about different group sizes in DD estimation? $\endgroup$ – DJJ Mar 20 '15 at 13:19
  • $\begingroup$ @DJJ unfortunately not from the top of my head but I guess searching for "power calculation"/"minimum detectable effect" together with difference in differences in your preferred search engine might turn up further useful references (guess google scholar would be your best bet) $\endgroup$ – Andy Mar 20 '15 at 13:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.