First, in randomized trials, all these methods eventually aim to estimate the same quantity: difference in post-treatment scores between two arms (our treatment effect), and the corresponding point estimators of treatment effect are all unbiased. This definition of treatment effect makes sense if you know pre-score is just a baseline covariate (not outcome) like other covariates. If you do not collect any baseline variables, what do you compare between two arms? Difference in post-treatment scores. We normally use ANOVA or two sample t.
So these methods are comparable and we can compare the variances of their treatment effect estimators and choose the method that yields the smallest variance estimator for its treatment effect estimator.
Second, it turns out ANCOVA main effect model is the most efficient approach with smallest variance estimator. That is, you fit a regular OLS regression in SAS
Model Post_score=group pre_score;
And you read the estimate and p-value associated with group variable. If you really want to repeated measures model, you should try a constrained repeated measure model (cRM) in which the main effect term for group should be excluded
Model Y=time time*group;
You report the estimate and p-value associated with “time*group”.
The reason why regular repeated measure model (RM) is less efficient is because it needs to estimate one extra parameter for the main effect of treatment group as follows:
Model Y=group time time*group;
You report the estimate and p-value associated with “time*group”. With the inclusion of “group”, RM assumes the mean pre-scores are different between two arms, which is generally not true in a randomized trial. That is why cRM forces the coefficient associated with group to be zero and remove this main effect group term from the model.
cRM and ANCOVA estimators of the treatment effect are equivalent, their variances are equivalent, and thus they have comparable performance. However, ANCOVA is more meaningful conceptually because pre-score is really a baseline covariate, not really an outcome. Outcome is something we observe after applying the treatment.
Change score is also less efficient. We fit a change score model in SAS like this:
We report the estimate and p-value associated with “group”. Change score model and regular RM model have the exactly the same point estimator of treatment effect. When there is no missing data, the two estimates are numerically same. Thus, both change score model and regular RM do not utilize the fact that pre-scores are the same between two arms.
ANOVA or two sample t test on post- score basically ignores the baseline information so it is least efficient in general situations.
Third, if there is an interaction effect. The situation is a bit more complicated. In previous discussion, we basically assume the pattern of association between pre-post scores is the same for everybody. When there is an interaction effect, it suggests the pattern of association between pre-post scores can be different between two arms. For example, subjects in the treatment group may respond very differently to treatment so their post-scores can vary a lot and the association between pre-post scores is generally weaker (than their counterparts in control group). We normally would expect the association between pre- and post-scores are stronger because placebo control would not change post-scores that much.
In this case, you can either fit an ANCOVA interaction model.
Model Post_score=group pre_score_meancentered group*pre_score_meancentered;
pre_score_meancentered is computed by subtracting overall sample mean pre-score from each individual pre-score. You can get the estimate associated with group. However, p-value is not correct (heteroscedasticity and extra variability in estimating sample mean are not accounted for). There is an adjustment in variance estimator you need to do.
Or you can still fit an ACNOVA main effect model
Model Post_score=group pre_score;
The main effect model is still OK even we need to fit an interaction term. The estimate associated with “group” is still unbiased. When the design is balanced with equal sample size in both arms, regular p-value is valid. In the unbalanced design, regular p-value is not correct. You need to use heteroscedasticity consistent variance (reported from SAS proc reg).
The two ANCOVA models are the same in balanced design. When design is unbalanced, ANCOVA interaction model performs slightly better with smaller variance estimate and smaller p-value. However, ANCOVA main effect is much simpler to implement. When sample size is relatively large, there is not much difference between two ANCOVA models.
Certainly, you can fit a cRM model with difference variance/covariance structure specified for each treatment arm, if you really like to treatment baseline covariate pre-score as an outcome. Anyway, It will perform comparably to ANCOVA interaction model.
So final recommendation is: use ANCOVA. It is easy to implement and it is the most efficient approach. Do heteroscedasticity check if there is a potential interaction effect. If so, use homoscedasticity consistent variance and associated p-value when design is unbalanced.
Senn S. Change from baseline and analysis of covariance revisited.
Kenward MG, White IR, Carpenter JR. Re: should baseline be a covariate or dependent variable in analyses of change from baseline in clinical trials? (Liu GF et al., Stat Med 2009; 28: 2509–30).
Wan.f Statistical analysis of two arm randomized pre-post design with one post-treatment measurement