7
$\begingroup$

I am confused about the mixed advice regarding controlling for baseline differences.

Would you always control for a baseline between groups difference on a particular variable or only if the variable correlates with the DV?

I am using SPSS and conducting Mixed Model analyses to evaluate an intervention.

Jeromy, I tried to answer by adding comment, but can't seem to make it work. To answer:

@jeromy-anglim: Well it is a group randomised design (schools randomised into intervention or waitlist control). However, participants from waitlist control schools (control condition) were sent an invitation to take part in a questionnaire study, whereas participants from the intervention schools were invited to take part in a parenting intervention. Hence, a slightly more distressed sample (despite efforts to avoid this by offering child participants a $30 voucher). In my case there are more boys in the intervention sample, and the intervention sample is on average 3 months younger. No other baseline differences for adolescent report. But baseline differences on almost all parent reported outcome variables (with the intervention group being more distressed).

$\endgroup$
4
  • 2
    $\begingroup$ In general both conditions would be necessary to make a difference on the estimated treatment effect. $\endgroup$
    – Andy W
    Jun 2, 2011 at 4:41
  • 2
    $\begingroup$ Has there been random assignment of cases to conditions? $\endgroup$ Jun 2, 2011 at 6:09
  • $\begingroup$ So it sounds like it deviates in two ways from the ideal randomisation; randomisation is occurring at the level rather than the individual level; and there is some evidence of differential take up. One more question: Do you have pre and post intervention DV measures or just post? $\endgroup$ Jun 2, 2011 at 8:31
  • $\begingroup$ @jeromyAnglim : I have pre and post data for both control and intervention group. $\endgroup$
    – user4269
    Jun 3, 2011 at 4:41

2 Answers 2

5
$\begingroup$

You might want to read the following article

  • Pocock, S.J. and Assmann, S.E. and Enos, L.E. and Kasten, L.E. (2002). Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practice and problems. FREE PDF
  • Check out the discussion of answers to this question on best practice in analysing pre-post intervention designs

A few thoughts, although I confess I'm not an expert on this:

  • If you had perfect randomisation, then the significance test without covariates would be fair, although there might be power benefits in including covariates. The more strongly the covariate is related to the DV, in general, the more it will reduce your error variance (which can increase statistical power assuming an true effect exists). This in combination with base-line group differences on the covariates will lead to greater differences between covariate adjusted and non-adjusted estimates of the intervention effect.
  • It sounds like you have some mild departures from randomisation, in that randomisation happened at a group-level, and there may have been some differences in the uptake of the experiment. I'd be particularly interested to know whether there are reasons to expect the groups to differ in their means on the DV at baseline. The degree to which departures from randomisation in the protocol are a problem is related to the degree to which it leads to systematically different groups.
  • In most applications that I've seen, pre-test measurements are likely to capture most of the potential effects of any baseline covariates.
  • I think the big issue is that if there are substantial baseline differences on the dependent variable, then it can be difficult to assess the effect of the intervention.
$\endgroup$
1
  • $\begingroup$ Thanks, I agree. I have tried to just focus on looking at the between group differences regarding slope (interaction between time and condition) and the estimated mean difference for each group (which I get asking for emeans table comparing) with confidence intervals. I am using multilevel mixed model analyses. My thoughts are that I can still describe the slope, however can't draw conclusions regarding between groups differences at follow-up. I think that Mixed Model deals with violations of homogeneity of regression slopes. $\endgroup$
    – user4269
    Jun 3, 2011 at 7:04
2
$\begingroup$

If your goal is to show group differences on an outcome (Y), there's no benefit to including a covariate unless it's relevant to (correlated with) Y. But you have to carefully think things through. There are times when controlling for a covariate serves to "equalize," making for a fairer comparison (in addition to increasing the precision of an analysis). There are other times when, regardless of any purely statistical results, attempting to equalize creates a bizarre scenario (and this relates to @Jeromy's last point).

Suppose we simplify by saying that controlling for a covariate entails adjusting each group so that it is treated as if it were average with respect to the covariate. But what if Group A could never be average on that variable without losing something that is essential to "Group A-ness"? This is a tricky area of statistics and a controversial one. Recipe-oriented textbooks--those that focus on procedures at the expense of concepts--tend to gloss over it. I'll bet the Pocock et al. article is a good one; I'd search around for others, or look into books by reflective authors like James Davis, Geoffrey Keppel or Elazar Pedhazur.

There's also good info on this topic at this thread

$\endgroup$
1
  • $\begingroup$ And if the groups are matched on certain characteristics, e.g. age, then there's no point adding those characteristics in as covariates. This tends to be an issue with case-control type studies, where one doesn't want to match on too many characteristics. $\endgroup$
    – Michelle
    Feb 10, 2012 at 11:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.