1
$\begingroup$

I have three groups with two hundred individuals each, and each individual within a group has a pre and post treatment measurement. A single group contains individuals with similar pre measurement scores.

If I wanted to see which groups have had a significant change in their measurements from pre to post treatment, what statistical test should I use?

$\endgroup$
1
  • 2
    $\begingroup$ Can you say more about how the groups were formed? $\endgroup$ Commented Dec 28, 2013 at 21:14

2 Answers 2

2
$\begingroup$

If I'm not mistaken, this is a pretty straightforward application for a mixed-design ANOVA, at least with the "group" trichotomy you've described. That would be your between-subjects factor, and the pre-vs.-post distinction in your repeated measurements would be your within-subjects factor. The $F$ test corresponding to this within-subjects factor will calculate the significance of change overall, and the test corresponding to the interaction of the between- and within-subjects factors will calculate the omnibus significance of differences across groups in the changes you observed.

You can then run post-hoc comparisons of your three groups to calculate the significance of specific group differences. You might also consider using contrasts (Rosnow & Rosenthal, 2002) if you have a particular theory or set of theories of differences that you want to test (e.g., a linear increase from group 1 through group 3, or no difference between two groups but an equivalent difference between the third and each other group). These generally aren't used on change scores though, so some adaptation might be necessary, and potentially even invalid due to problems with simple change scores (leptokurtosis, for instance).

BTW, it kinda sounds like you've chosen to impose a trichotomy on your total sample based on differences in pretest scores. This is a questionable choice if there aren't natural differences between your three groups; if the pretest scores for your total sample are normally, unimodally distributed, for instance, you're probably making very artificial splits in your data that may lead to misinformed inferences, or at least a loss of information. If you don't have other reasons to divide your sample into these three groups, you might want to just look directly at the relationship between change and the pretest scores in their original (continuous, I assume; not trichotomized) form. If that's all you really need to do, you can do it with a latent change regression (McArdle, 2009), which can describe the relationship between baseline and change in a continuous variable over time. If this variable is actually a scale score (e.g., sum or average of individual question responses) or latent factor with multiple indicators, incorporating that measurement model in the structural equation model would even remove the leptokurtosis problem with simple change scores. That won't be the case if there is error in the measurements that you can't remove though (e.g., if your variable of interest is a single, simple measurement, and not based on multiple indicators). Of course, if you do have a strong reason for dividing your total sample into three discrete categories, you can disregard this last paragraph entirely. I'm just suspicious about this because you said, "A single group contains individuals with similar pre measurement scores," and "similar" sounds like a subjective judgment about a continuous variable, which is a dubious basis for artificial categorization. If I'm going off on an unnecessary tangent here, let me know; I can just edit this out.

References

McArdle, J. J. (2009). Latent variable modeling of differences and changes with longitudinal data. Annual Review of Psychology, 60, 577-605.

Rosnow, R. L., & Rosenthal, R. (2002). Contrasts and correlations in theory assessment. Journal of Pediatric Psychology, 27(1), 59-66. Available online, URL: http://jpepsy.oxfordjournals.org/content/27/1/59.full.pdf.

$\endgroup$
3
  • 1
    $\begingroup$ Agree with the last paragraph. The question in the OP is simply answered by running three paired t tests (or similar), however. $\endgroup$
    – Michael M
    Commented Dec 30, 2013 at 8:07
  • $\begingroup$ That occurred to me as well! Wouldn't it inflate the chance of false alarm error to conduct three separate t-tests though? That is, if three paired-samples t-tests were sufficient, wouldn't it also be necessary to adjust the alpha level for each of the three individual retain/reject decisions, assuming the intention is to conform the overall analysis' alpha to the conventional "reject the null if p < .05" rule? (Ideally, one would probably want to handle that decision differently too, but that's a slightly separate issue and a significantly longer argument.) $\endgroup$ Commented Dec 30, 2013 at 8:23
  • 1
    $\begingroup$ This is definitively to be thought of. (As in the post-hoc tests on the repeated measures anova). If an type 1 error rate of 5% is acceptable per group, then we don't need to correct for multiplicity. If the total type 1 error is to be bounded, the we can use Bonferroni-Holm correction etc. $\endgroup$
    – Michael M
    Commented Dec 30, 2013 at 8:33
-1
$\begingroup$

Take log ratios of post/pre measurements for each individual within each group obtaining 3 vectors. (The ratios to make sure that the differences between pre and post measurements are independent of size of measurements in each group. And log to make the ratios symmetricaly spaced as for increase and decrease.) Then perform the Kruskal-Wallis H test on that 3 vectors (which is to test the hypothesis that the three groups don't differ significanly (provided you can't assume normality)).

$\endgroup$
2
  • $\begingroup$ You should explain the reasoning that lays behind this advice, including Why logs? and Why the Kruskal-Wallis test?. $\endgroup$
    – Glen_b
    Commented Dec 28, 2013 at 22:46
  • $\begingroup$ The question is about comparing pre and post measurements, not groups. $\endgroup$
    – Michael M
    Commented Dec 29, 2013 at 13:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.