1
$\begingroup$

I have a continuous variable, and a small sample of 8 observations at 4 different time points (before some intervention, and at 3 follow-up time points). My sample, being a preliminary experiment, does not give me enough power to run a mixed model or a GEE model. I see graphically that there are differences at all 3 time points, compared to baseline (a decrease of the response). When I run a mixed model, only the 1st follow-up is significantly different than the baseline, with the 2nd having a p-value of 0.06 and the 3rd, with a smaller effect, is not significant (can be seen graphically that the effect is smaller). I tried running a paired t-test separately for each time point (compared to baseline). The first 2 test were statistically significant (as the graph suggested) and the 3rd not (although the graph does show some effect). My question to you is, should I apply some multiple hypothesis testing correction in this case ? Using the Bonfferoni correction doesn't change the outcome, still 2 tests are signigifant. I just want to do it properly. I am quite confident that with an addition of a few samples also the 3rd time point would have been significant. Is my analysis valid at all ?

$\endgroup$
3
  • $\begingroup$ what I meant to say is that with an addition the mix model would give significant result. With more parameters to estimate, the mix model doesn't have enough power with n=8 $\endgroup$
    – user61747
    Nov 30, 2014 at 9:17
  • $\begingroup$ Do you have complete data at all time points? Hopefully you do since it is an experiment! $\endgroup$
    – Moose
    Nov 30, 2014 at 12:26
  • $\begingroup$ Yes, I do. I have 8 subjects and 4 time points for each (including baseline). 32 data points. $\endgroup$
    – user61747
    Nov 30, 2014 at 12:28

1 Answer 1

1
$\begingroup$

It is not that you have enough power for mixed models or GEE, but rather than you do not have a high enough sample size to confirm if your data/models meet all of the assumptions. Or the model estimation might not even converge on a solution.

Your best bet would (probably) be to run a repeated-measures ANOVA, then following a significant omnibus test, perform your pairwise comparisons. Apparently there is some debate about how to do this for repeated samples because of the between-group dependence. You certainly would not be committing an error if you performed multiple paired t-tests and adjusted for multiple comparisons (Holm will be better than Bonferroni because it always has more power; see the multcomp package in R).

Also, if you change over time is relatively simple (e.g. linear), you could consider doing a regression.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.