Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I'm analyzing a double-blind, placebo controlled RCT using an ANCOVA in R, where the predicted "treatment effect" shows up before the treatment has occurred!

The goal of the study is to determine if treating a disease reduces a particular behavior. Folks with both the disease and the behavior were randomized 50-50 into treatment and placebo control arms.

As predicted there was a significant interaction between the intervention and baseline disease state (all variables are interval, except for "intervention", which is a two-level factor variable identifying treatment vs. control group):

MODEL 1:

Call:
lm(formula = follow_up_behavior ~ baseline_behavior + baseline_disease * 
intervention, data = d)

                                   Estimate Std. Error t value Pr(>|t|)    
(Intercept)                            11.28464    0.51177  22.050   <2e-16 ***
baseline_behavior                       0.77872    0.05112  15.234   <2e-16 ***
baseline_disease                        0.36726    0.23110   1.589   0.1146    
interventiontreatment                   0.74738    0.70254   1.064   0.2895    
baseline_disease:interventiontreatment -0.64681    0.31374  -2.062   0.0414 *  

The interaction was predicted because the intervention is a very effective treatment of the disease, but baseline disease state varies along a continuum from near 0 to very high. Hence, folks with high baseline disease got the biggest benefit from the treatment, and should therefore have had the biggest reduction in the behavioral response. So far so good.

On a lark, I ran a very similar model of baseline behavior:

MODEL 2:

Call:
lm(formula = baseline_behavior ~ baseline_disease * intervention, 
data = d)
                                        Estimate Std. Error t value Pr(>|t|)  
(Intercept)                             -0.6350     0.7620  -0.833   0.4062  
baseline_disease                         0.7422     0.5016   1.480   0.1415  
interventiontreatment                    1.1941     1.0611   1.125   0.2626  
baseline_disease:interventiontreatment  -1.3320     0.6510  -2.046   0.0428 *

As you can see, there is a very similar significant interaction between disease state and intervention group, even though though the intervention has not yet occurred. This would seem to be a failure of randomization.

My primary concern is that because follow_up_behavior is very highly correlated with baseline_behavior, the significant interaction in the first model is due to the pre-existing interaction seen in the second model, and is therefore not a consequence of the intervention.

My questions are:

  1. Do I actually have a problem?

  2. If so, does including baseline_behavior as a control variable in model 1 fix the problem, i.e., guarantee that the significant interaction in this model is not a consequence of the pre-existing interaction seen in model 2 but is instead due to the intervention?

  3. If including baseline_behavior as a control is insufficient, is there anything I can do salvage the study?

Many thanks in advance for any help or insights.

share|improve this question
    
Jeromy, thanks for improving my title. –  biomarker Jun 8 '12 at 21:47
add comment

1 Answer

If there is enough data to do this include the most significant baseline covariates in the model giving you a way to adjust for covaraite imbalance. There is an interesting book by Vance Berger that specifically addresses the issue of covariate imbalance in clinical trials and how to detect it.

share|improve this answer
    
Thanks. The reference to Berger is very helpful. He has quite a few articles on the topic, and these have lead me to useful articles by other authors. –  biomarker Jun 8 '12 at 21:32
1  
But, I do have the most significant baseline covariates in the model. What's throwing me is that the imbalance in baseline behavior is in interaction with baseline disease. Berger mentions in passing that one theoretical approach he uses applies to interactions (at least, I think that's his point), but it's over my head. In general, there seems to be a lot of discussion of covariate imbalance, e.g., too many old folks or too many men, but not of interacting imbalances, e.g., too many old men. –  biomarker Jun 8 '12 at 21:37
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.