Perhaps I am missing something obvious here, but I can't find an answer to this. Let's say we run 10 individual repeated measures models with factor(time) as fixed effect with 5 separate timepoints and 10 different outcome variables. You would get p-values of time 0 vs 1, 0 vs 2, etc. for each outcome variable. Since you have used 10 models how would you correct for multiple testing? And how would that affect p-values between the individual time points?
Strictly speaking you will need to correct for both the multiple comparison within and between outcomes. However, this may be an overkill at the end. Often what is done it is done in such situations is the following:
- For each outcome you first test with an omnibus test whether there is any different between the time points. If this omnibus test is significant, then you perform the post-hoc tests to see which time points different with each, correcting for multiple testing within the outcome.
- Then for the different outcomes, you typically pre-select a couple as your primary outcomes and the rest as secondary outcomes. To be fair, this selection needs to be done a-priori and without looking at the results first. Then you only correct for any multiple testing you have in the primary outcomes (i.e., if you have selected more than one). For the secondary, you report the results without correction. You need to be clear and explicit in your text that you have not corrected for multiple testing in the secondary outcomes. And hence, you need also to be more careful on the formulation of statements based on the results from these outcomes.