Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Why would one want to control for any number of baseline covariates in a situation where the assignment to treatment group is random?

My understanding is that randomly assigning treatment should make the treatment variable strictly exogenous, creating a control group that can appropriately be considered as a counterfactual. The only exception I can think of is when sample sizes are small, and that random assignment can still produce unbalanced groups.

Any thoughts are much appreciated. Thanks!

share|improve this question
add comment

2 Answers

up vote 2 down vote accepted

From a frequentist perspective, an unadjusted comparison based on the permutation distribution can always be justified following a (properly) randomized study. A similar justification can be made for inference based on common parametric distributions (e.g., the $t$ distribution or $F$ distribution) due to their similarity to the permutation distribution. In fact, adjusting for covariates—when they are selected based on post-hoc analyses—actually risks inflating the Type I error. Note that this justification has nothing to do with the degree of balance in the observed sample, or with the size of the sample (except that for small samples the permutation distribution will be more discrete, and less well approximated by the $t$ or $F$ distributions).

That said, many people are aware that adjusting for covariates can increase precision in the linear model. Specifically, adjusting for covariates increases the precision of the estimated treatment effect when they are predictive of the outcome and not correlated with the treatment variable (as is true in the case of a randomized study). What is less well known, however, is that this does not automatically carry over to non-linear models. For example, Robinson and Jewell [1] show that in the case of logistic regression, controlling for covariates reduces the precision of the estimated treatment effect, even when they are predictive of the outcome. However, because the estimated treatment effect is also larger in the adjusted model, controlling for covariates predictive of the outcome does increase efficiency when testing the null hypothesis of no treatment effect following a randomized study.

[1] L. D. Robinson and N. P. Jewell. Some surprising results about covariate adjustment in logistic regression models. International Statistical Review, 58(2):227–40, 1991.

share|improve this answer
    
Hello - interesting answer. Any interest in having an offline conversation about this? –  rolando2 Jan 25 at 17:51
add comment

If the outcome depends on treatment as well as other observable factors, controlling for the latter often improves the precision of the impact estimate (i.e., the standard error of the treatment effect will be smaller). When sample size is small, this can be helpful.

Here's a simple simulation where even though treatment is random, the standard error shrinks by a third:

. set obs 100
obs was 0, now 100

. gen treat =mod(_n,2)

. gen x=rnormal()

. gen y = 2 + 3*treat + 1*x + rnormal()

. reg y treat

      Source |       SS       df       MS              Number of obs =     100
-------------+------------------------------           F(  1,    98) =  112.75
       Model |  209.354021     1  209.354021           Prob > F      =  0.0000
    Residual |  181.973854    98  1.85687606           R-squared     =  0.5350
-------------+------------------------------           Adj R-squared =  0.5302
       Total |  391.327875    99  3.95280682           Root MSE      =  1.3627

------------------------------------------------------------------------------
           y |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       treat |   2.893814   .2725345    10.62   0.000     2.352978     3.43465
       _cons |   2.051611    .192711    10.65   0.000     1.669183     2.43404
------------------------------------------------------------------------------

. reg y treat x

      Source |       SS       df       MS              Number of obs =     100
-------------+------------------------------           F(  2,    97) =  180.50
       Model |  308.447668     2  154.223834           Prob > F      =  0.0000
    Residual |  82.8802074    97  .854435127           R-squared     =  0.7882
-------------+------------------------------           Adj R-squared =  0.7838
       Total |  391.327875    99  3.95280682           Root MSE      =  .92436

------------------------------------------------------------------------------
           y |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       treat |   2.918349   .1848854    15.78   0.000     2.551403    3.285295
           x |   1.058636   .0983022    10.77   0.000     .8635335    1.253739
       _cons |   1.996209    .130825    15.26   0.000     1.736558     2.25586
------------------------------------------------------------------------------
share|improve this answer
1  
+1 - Another reason is to identify interaction effects with treatment, but that takes more than just "controlling" for other factors like the OP mentioned. –  Andy W Jan 24 at 23:55
    
Thanks! So, if a one or more covariates affect the outcome you're trying to measure, including them in your model will improve the precision of your estimate of the randomly assigned treatment effect, but won't really impact your estimate of the value of the treatment coefficient, correct? –  Robb Jan 26 at 2:57
    
Yes, that's right. –  Dimitriy V. Masterov Jan 26 at 18:01
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.