I am carrying out a difference in difference estimation. Regarding control variables addition I am kinda confused. Am I to add control variables which affect the dependent variable or control variables which affect the dependent variable but would not affect the policy effect on the dependent variable.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
5
There are two reasons for including covariates in a difference in differences regression:
- for identification of the treatment effect;
- to reduce the error variance (i.e. increase power of statistical tests).
Suppose you want to know the effect of a job market program on employment in a city where this program was randomly assigned to unemployed individuals. You regress $$Y_{igt} = \gamma_g + \lambda_t + \beta J_{gt} + \epsilon_{igt},$$ where $Y$ is the outcome (0 = unemployed, 1 = employed), $J$ is the dummy which equals one for the treated in the treatment period, $i$, indexes individuals, $g$ indexes groups (1 = treatment group, 0 = control group), $t$ indexes time periods (1 = post-treatment, 0 = pre-treatment), $\gamma$ and $\lambda$ are group and period fixed effects, and $\epsilon$ is a stochastic error term.
This is equivalent to the regression $$Y_{igt} = \alpha + \beta_1 \text{treat}_i + \beta_2 \text{post}_t + \beta_3 \text{(treat$\cdot$ post)}_{it} + \epsilon_{it},$$ where you indicate treatment and control group, and the pre- and post-treatment periods with two dummies that are then interacted. This is probably the difference in difference regression you have seen in a textbook or class notes.
Suppose in the two periods the GDP in our city increases. Then the outcome is affected by another time-varying factor which has nothing to do with the job market program but it occurs at the same time. This violates the common trend assumption. In order to not overstate $\beta$ in the first regression, you would want to include GDP as a control in the regression which separates out this additional effect on the outcome which is not due to the job market program. This is the case of including a covariate for identification when you regress $$Y_{igt} = \gamma_g + \lambda_t + \beta J_{gt} + \delta \text{GDP}_t + \epsilon_{igt}.$$
Only information at the group level (treatment-control) is required for identification of your treatment effect. If you have additional information on your individuals, for instance their age, you can include this in the regression as well, as follows: $$Y_{igt} = \gamma_g + \lambda_t + \beta J_{gt} + \delta \text{GDP}_t + \theta \text{age}_{igt} + \epsilon_{igt}.$$ The within-group variation in age does not matter for identification (for a detailed explanation see the notes here) but they can help to reduce the error variance of the regression. In this case your statistical tests will have more power and it will be easier to determine whether $\beta$ has a significant effect or not.
-
$\begingroup$ Alrite. I think I understand the write up. Though I am using repeated cross sections and not panel data. One region received the treatment and the other did not. So I defined treat=1 fot the region received the treatment and treat=0 for the region that did not, meaning that in the pre-treatment period the region that received treatment had 1 and the other region had 0. I defined post=1 if in post treatment period and post=0 if in pretreatment period. $\endgroup$ Commented Aug 6, 2014 at 15:36
-
$\begingroup$ From the question I could only guess what you want to do, so I provided an own example with the job training program. Difference in differences may rely on too strong assumptions when you use repeated cross sections rather than panel data (see here). Otherwise I hope the answer explained the different uses of control variables in this method. $\endgroup$– AndyCommented Aug 6, 2014 at 15:40
-
$\begingroup$ Actually my reply got posted by mistake before completing my comment. What Im jst confused abt is that usually without performing a diff-in-diff analysis, there are variables that determine the dependent variables. I was wonderin if I shld include them along side treat, post and interaction. $\endgroup$ Commented Aug 6, 2014 at 15:50
-
$\begingroup$ If they vary over time, then you should include such variables. Otherwise the diff in diff interaction coefficient will wrongfully pick up the effect of these variables because they also change at the same time as the intervention. $\endgroup$– AndyCommented Aug 6, 2014 at 15:57
-
$\begingroup$ While the previous reply was excellent, I think it needs to be highlighted that time-varying covariates are highly problematic as they are likely to be bad controls. $\endgroup$ Commented Oct 18, 2023 at 18:23