1
$\begingroup$

I am really hoping you can shed some light on a topic that I am struggling with.

I have a 2x3x4 repeated measured anova. I have a significant 3-way interaction, and I want to make sure that I am using the correct post hoc comparisons and not violating any key statistics theory.

If we consider my 2x3x4 RM ANOVA to be AxBxC. With the significant 3-way interaction I have performed the following post-hoc analysis:

AxB at each level of C

AxC at each level of B

BXC at each level of A

Is it typical for post-hoc tests on a three way interaction to perform all of the above 3 comparisons or would one typically just pick one or 2? They are all an important part of the story.

Also regarding adjustment for multiple comparisons, I have used SIDAK to adjust for for this within each of the above 3 tests. I am assuming that I do not need to adjust each p value further to account for the total number of comparisons in all 3 of the above?

Finally, most stats textbooks state that if you have an interaction term you can't reliably interpret the main effects. If I only have an interaction term for AxB, not AxC or BxC, is it still ok to interpret the main effect? Thanks for you help,

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
3
$\begingroup$

You should pick which slices to look at based on what is of interest to you; you are not violating statistical theory.

As for multiple comparisons - as Jacob Cohen put it in his book on ANOVA and regression - this is a topic about which reasonable people can differ. Some people think you don't need to correct for them at all. Others think you should do what you did. Others think you should correct for all the analyses involving the same variables. Remember that by adjusting for multiple comparisons you are lowering power

In essence, it's not a statistical question, it's a philosophical one. I mean, if you really don't want to ever make a type I error, should you adjust for all the tests you have run in your life? Or what?

As to your last question: If C is not involved in an interaction, you can report its main effects.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for the reply, that really helps. I thought that it wasn't so cut and dry. Do you have any suggestions for the final question I had about interactions and subsequent interpretation of main effect? $\endgroup$
    – user28327
    Jul 22, 2013 at 15:01
  • $\begingroup$ I edited my answer $\endgroup$
    – Peter Flom
    Jul 22, 2013 at 15:06

Not the answer you're looking for? Browse other questions tagged or ask your own question.