0
$\begingroup$

I'm currently reviewing a colleague's paper, and they have an intervention study (2x3 design) with 2 groups (therapy vs waitlist control) and 3 time points (pre-test, post-test, long-term follow-up). They are looking at several outcome measures.

For the data analysis portion, they simply calculated changes in scores between pre-intervention scores and post-intervention scores (e.g. pre minus post). They also calculated change scores for pre-intervention scores and 12-week follow-up scores (e.g. pre minus 12-week follow-up). They used independent t-tests to find significance of between group change scores (i.e. to see if the pre-post change score was significantly greater for therapy group compared to the waitlist control) and paired sample t-test for change within group.

I think a repeated-measures ANOVA or mixed effects model would be far superior to this method, but i'm having trouble explaining coherent the rationale behind this. I would love to hear thoughts on this...thanks so much in advance!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

You could always appeal to authority (while not being a very good argument can still be appealing) by referencing this paper by Bland and Altman on how you really should do an ANCOVA (that is, adjust for baseline in your linear model) instead of t-testing everything to death. Bland makes similar arguments in his book "Introduction to Medical Statistics" -- adjust for baseline and don't look at raw differences.

Aside from that, if your colleagues don't already realize that some sort of mixed model or longitudinal model is more appropriate, bombarding them with statistical complaints about within patient variance or correlation is not going to help. In my experience, you should lead them to the conclusion that other models are more appropriate and allow them to arrive at it themselves. If they don't want to (or are incapable of doing so) then that is a different problem.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for the quick reply Demetri! A quick follow-up. Would you be able to expand a bit on why the mixed model or longitudinal approaches would be better here? I'm a bit rusty on my data analysis, but off the top of my head I would say because they are able to account for more variance, limit the type-1 error (by running fewer t-tests), adjust for baseline differences, model the changes over time and be able to explore interactions, and have the ability to control for other variables. Is there anything you would add or change to that explanation? $\endgroup$
    – Lucidmind
    Commented Dec 17, 2019 at 22:27
  • $\begingroup$ @Lucidmind That all sounds well and good. You could add that the t-test would make the assumption that the points in time are completely independent of one another. We know that isn't the case (or at the very least have strong reasons to believe that is not the case). $\endgroup$
    – Demetri Pananos
    Commented Dec 17, 2019 at 23:17
  • $\begingroup$ awesome, thanks so much! $\endgroup$
    – Lucidmind
    Commented Dec 19, 2019 at 21:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.