# Bonferroni and Greenhouse-Geisser corrections with repeated-measures ANOVA

I guess my question comes in two parts:

1. Following a 1x7 repeated-measures ANOVA, I carried out 3 pre-planned tests. Two were t-tests between 2 of the conditions. The third is a 1x6 ANOVA (to test significance between conditions 2-7). My question is:
Do I have to apply the Bonferroni correction to all 3 tests, or just the 2 tests? I.e. should the significance level be 0.05/3 = 0.017 for all 3 tests?

2. As above, I carried out a 1x7 repeated-measures ANOVA, and 3 pre-planned tests (2 t-tests and a 1x6 ANOVA). In the original 1x7, sphericity was assumed, so I report the 'sphericity-assumed' statistic, and it is non-significant. Next, despite this non-significance, I decided I should do the pre-planned tests for the sake of completeness. When I do the 1x6 repeated measures ANOVA, sphericity is NOT assumed, so I use the Greenhouse-Geisser values, and it comes up significant (if I were to use the shericity assumed values, it would be non-significant). My problem here is that the ANOVA using just 6 of the 1x7 conditions is coming up significant when the original 1x7 was non-significant. Logically, this doesn't seem to make sense to me.
Is there some 'rule of thumb' whereby the use of Greenhouse-Geisser values needs to be consistent?

I hope this makes sense. I have trawled the internet. I have looked at more stats books than you can shake a stick at. But nothing comes close to discussing this sort of situation.

-
Re: your first question, multiple comparisons is a frequently discussed topic on this site, so I'd suggest do a search of the old threads. I can't comment directly on your second question (re: sphericity), but it certainly is possible, in ANOVA, to reject the null in the first case, but not reject the null when you've added a new level - particularly if that new level has a mean equal to, say, the mean of the other 6 levels. –  Macro Apr 26 '12 at 12:12
Re: Marco... I see your point about adding a new level bringing significance to non-significance, but in my data, removing a level turned non-significance to significance... –  Eric Apr 26 '12 at 12:53
Maybe the level that was removed had a mean very similar to the others so, by removing it, there was more heterogeneity in the means? –  Macro Apr 26 '12 at 12:55

Regarding Bonferroni (and multiple comparisons issues in general) Jacob Cohen, in his book on regression, said "this is a subject on which reasonable people can differ". There are arguments for not doing such corrections at all (see, e.g., this piece by Andrew Gelman). I find such arguments persuasive.

If you reduce chance of type 1 error then (other things being equal) you increase chance of type 2 error (that is, you reduce power). In many fields, the "given" values are .05 for type 1 and .8 for power (.2 for type 2). But these may or may not be sensible. Type 2 error may be much worse than type 1 error.

In my view, the important thing is not statistical significance, and certainly not whether it is below .05, but effect size and precision of effect size.

If you correct for multiple comparisons, it is really a question of what "multiple" is. Paraphrasing Cohen, should it be for one experiment? One article? One set of results? Results about one topic? All results you ever do? Or what?

From a purely practical point of view, you may be required to do some particular thing to satisfy journal editors or other supervisors.

-
+1 Interesting points, thanks. –  Freya Harrison Apr 26 '12 at 12:49