Use of "post-hoc" tests - do these require an ANOVA to be performed first? [duplicate]

Question

I would like to compare multiple means using a method such as an ANOVA. This would have to be followed up with either Scheffe's test or the Games-Howell post-hoc method (my data is non-normal - to different degrees and can't all be transformed to normality using the same transformation - heteroscedastic and has greatly differing sample sizes).

In their Opthalmic and Physiological Optics paper, Armstrong et al. (2000) state that these tests can only be performed after an ANOVA returns a significant result. My question is this - how can an ANOVA be appropriately applied to data such as mine, which require these post-hoc tests simply because they violate so many assumptions of the ANOVA? It seems odd to me that they should advocate the use of an ANOVA before post-hoc tests that are used in place of others because of the violations of the ANOVA assumptions.

Also - on a side note, does anyone have any advice as to whether to use the above approach or to use the multcomp package available in R?

Many thanks!

Answer 1

While it is true that some post hoc tests require an ANOVA for justification, like a Fisher's protected LSD, there are others for which the significant ANOVA precursor is not really required. Nevertheless, they're called post-hoc tests for a reason. :)

You can just do independent tests and correct for multiple comparisons. If you plan out exactly which comparisons you wish to make first and correct for only those you'll be better off. You may want to look at the wikipedia page on multiple comparisons to see what method best fits your situation.