ANCOVA or repeated measures ANOVA?

I have 3 groups randomised to intervention A or B or control. Participants were measured pre and post intervention - should I use repeated measures ANOVA or ANCOVA with the variable measure at time 1 (pre) as the covariate? Variables are continuous. thanks

-
Sheila, What did you go with in the end? I am currently trying to decide the exact same thing (only with a 2x2 design, experiment group vs. control group at Time1 and Time2). My numbers are small (n=19 in experiment and n=17 in control) so need to bear that in mind - plus the fact that at least one of the DVs is not normally distributed... Thanks! –  user13833 Sep 4 '12 at 14:00

I would generally say that you need a repeated measures model with group, pre_post, and an interaction term. What you want to know is whether the post-test score is dependent upon the intervention, so you need to see an interaction. The meaning of that interaction would depend on actual scores.

A simpler method might be to use a pre_post subtraction score. That would make the assumption that pre-test differences (significant or otherwise), do not affect the outcome at all, or are not present at all. Check your groups, and if they are very highly similar then the subtraction score and one-way ANOVA is an easy thing to do.

The ANCOVA is much more like the difference scores or also an ANOVA on the residuals left after removing pre-test. It's not wrong, and some people prefer it. I believe I read a paper once recommending it but even they qualified it because pre-test may be correlated with group. In that case interpretation becomes difficult.

Why not run the full ANOVA and the ANCOVA? If they reach similar conclusions you're safe. If they reach different conclusions then think much more about what your data mean and maybe come back on here and ask for help at the interpretation stage.

-
thnx - see my response below –  Sheila Mar 25 '12 at 15:49
I think that the F test for the interaction term in ANOVA with group and pre_post as factors yields exactly the same result than a one-way ANOVA on difference scores. Whatever the merits of this approach compared to an ANCOVA, it seems that it does not matter whether you choose for an independent sample t-test (in the two-group case) or one-way ANOVA on gain scores vs. a 2x2 mixed ANOVA. At the end of the day, comparing this and an ANCOVA seems like a good strategy however (+1). –  Gaël Laurans Jun 10 at 13:07
The interaction term and pre_post should be the same, yes. But, by failing to analyze the pre effects you've hidden potential interpretation issues. It's better to have them all clearly in the open. –  John Jun 10 at 13:26
Well, if treatment assignment is properly randomized, it does not make sense to test for a difference. In any case, there is no additional assumption in the change score approach. If you are prepared to interpret the interaction term, you might just as well run the one-way ANOVA directly. –  Gaël Laurans Jun 10 at 15:14
I'm not sure what you mean by "properly randomized". Random assignment can result in baseline differences by random chance and those can impact the interpretation of subsequent difference scores. –  John Jun 10 at 20:12
show 4 more comments

I suspect the best approach would be to get difference scores (i.e., post - pre) for each experimental unit, and then run a simple one-way ANOVA.

-
It may be the easiest but I don't think characterizing it as the "best" is reasonable. At the onset it precludes seeing if any baseline*treatment interaction exists, and as John mentions it is problematic if the baseline has a causal effect on the post-treament (independent of treatment itself). Paul Allison in this article gives some examples where change scores are preferable. –  Andy W Mar 25 '12 at 15:24
this is very helpful _ i have already done a repeated measure and have a sign interaction but contrasts did not reflect this . so i looked at a change score (ie i made the new variable of the difference) and put this in a simple ANOVA and it showed where the signficiant differences were but I am worried that this is poor statistical methodology and that all the testing should be within one test?? maybe this isnt right - i think on a quick look that ANCOVA may show this significance but dont want to inflate the type 1 error. i dont know if group is correlated to outcome - i'll look at this –  Sheila Mar 25 '12 at 15:48
@AndyW A lot of this discussion focuses on situations where people cannot be randomly assigned to the condition. Not to say that it's entirely irrelevant but that's something to keep in mind when trying to relate it to the present question. –  Gaël Laurans Jun 10 at 12:52
Sure @GaëlLaurans, the refence I gave in my comment to the Paul Allison article provides data generating process examples where change scores are preferable. I disagree they should be a default though, that was my only point. –  Andy W Jun 10 at 13:11