Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have two groups of 10 participants who were assessed three times during an experiment. To test for differences between groups and across the three assessments, I ran a 2x3 mixed design ANOVA with group (control, experimental), time (first, second, three), and group x time. Both time and group resulted significant, besides there was a significant interaction group x time.

I don't know very well how to proceed to further check for the differences between the three times of assessments, also respect to group membership. In fact, at the beginning I only specified in the options of the ANOVA to compare all the main effects, using the Bonferroni's correction. However, then I realized that this way they were compared the differences in time of the total sample, without group distinction, am I right?

Therefore, I searched a lot on the internet to find a possible solution, but with scarce results. I only found 2 cases similar to mine, but their solutions are opposite!

  1. In an article, after the mixed design, the authors ran 2 repeated measures ANOVA as a post-hoc, one for each group of subjects. This way, the two groups are analysed separately without any correction, am I right?
  2. In a guide on the internet, they say to add manually in the SPSS syntax COMPARE(time) ADJ(BONFERRONI), just after /EMMEANS=TABLES(newgroup*time), while running the mixed ANOVA. This way, the three times are compared separately for each group, with Bonferroni correction, am I right?

What do you think? Which would be the correct way to proceed?

share|improve this question
Winer (1962)'s master text on stats. provides formulae for the errors terms to be used in post hoc comparisons following many kinds of ANOVA, including this one. – user41999 Mar 16 '14 at 0:36
Hi @StuartMcKelvie, could you give more details? As it stands, your answer is hardly usable by the OP or future visitors. (Plus, you don't provide a reference for Winer [1962], and since it's so old, it might not be easy to find.) – Patrick Coulombe Mar 16 '14 at 1:03
I found this free chapter from IBM SPSS Statistics (18&19): Workbook It's exactly about your case – sviter Aug 4 '14 at 14:06

I don't know SPSS syntax particularly well, but, if I understand your situation correctly, the significant interaction means that, in order to adequately assess the significance of your main effects, you'll need to do separate analyses. I think the best way to proceed is to do separate repeated measure analyses for each level in your grouping factor. Perhaps someone else can speak better to the question of how to handle correcting for multiple comparisons during post-hoc analysis, but I'm pretty sure you still need to use a correction. You might try Tukey's, as a multiple comparison correction!

share|improve this answer
Thank you for your answer. If I understood correctly, you suggest the solution 1), to conduct two separate repeated measures ANOVAs, one for each group, with time as the within subjects indipendent variable (3 levels) and then, if significant, compare the main effects with Tukey's correction (or Bonferroni, I guess, isn't it ok?). Have I understood correctly? – Federico Nov 18 '12 at 11:55
In this case, using SPSS I selected "Data/Split file..." and entered the grouping variable. Is this correct? This way, I found a slightly significant ANOVA (p = 0.044) for the control group, but Bonferroni (it doensn't allow me to do Tukey) comparisons are all non significant... How should explain this? Is the ANOVA result a I type error? – Federico Nov 18 '12 at 11:57

In short. There is no globally accepted convention for these situations. Some will use Bonferroni corrections. Some will force the Tukey HSD framework to dance for them (e.g. Maxwell & Delaney). In contrast...

COMPARE(time) ADJ(BONFERRONI)", just after "/EMMEANS=TABLES(newgroup*time)

... does seem to use the Bonferroni correction. However, this approach is likely conservative, especially in the face of Holm-Sidak style corrections. (ESPECIALLY if you don't use the MSW as the error term for your post-hoc comparisons).

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.