There seems to be some variability in WHEN multiple comparison tests such as Bonferroni or FDR might be implemented.

Type 1: One might carry out multiple one-way ANOVA's to see whether or not there is a significant interaction term. Some studies apply the adjustments at this level and determine whether they can even move on to follow-up tests such as pairwise comparisons.

Type 2: I have also seen (probably more of) others wait to apply the adjustments (BF, FDR) until after they note a significant one-way ANOVA from multiple ANOVA's. They would then apply adjustments to the pairwise contrast. They might have already done several ANOVA tests to get there without having applied the corrections.

Question 1: Is one type (1 vs 2) of practice more acceptable than other?

Question 2: How might one justify proceeding with type 2 if there were several ANOVA's that were required to get to the pairwise comparisons? (reasons for skipping this aren't often mentioned in manuscripts I have seen)

Question 3: I realize there is no strict rule of thumb with HOW MANY tests require correction. Nonetheless, people seem to have preferences or generally agreed upon threshold (e.g., 5 or less is okay, 10 or less might be borderline, 15 or above definitely requires corrections...)

Question 4: Instead of using several anova (or manova), you go into SEM to use a path analytic approach. Does this seem feasible and would it bypass some of the issues that multiple ANOVA's raise?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.