0
$\begingroup$

I have a quick question about reporting simple main effects. I am running a two-way mixed ANOVA and the interaction effect was not significant. I understand that if you have no significant interaction effect, you should not carry out a test of simple main effects and instead just interpret the main effects. I really don't want to do this, the main effects give me nothing. So I carried out the test of simple effects anyway (just to see) and they are exactly the results I was expecting/looking for, and now I'm doubting my source which claims you need a significant interaction to run test of simple effects? Any help would be greatly appreciated.

$\endgroup$
  • $\begingroup$ Could you please clarify your distinction between "simple main effects" and "main effects"? $\endgroup$ – EdM Jul 10 '16 at 18:21
  • $\begingroup$ @EdM A simple main effect is an effect of a variable considering only the data at one level of another variable (as opposed to across all levels, as in a main effect per se). $\endgroup$ – Kodiologist Jul 10 '16 at 22:16
1
$\begingroup$

Generally, when somebody says that data must satisfy some condition C (e.g., they must produce a significant interaction term in an ANOVA) before you can use a follow-up procedure (e.g., tests of simple main effects), what they mean is that any guarantees that the follow-up procedure will be accurate or useful require C. So, if C does not hold, you have lost your reason to believe anything the follow-up procedure tells you. If doing the follow-up test gets you results you wanted anyway, that's no reason to believe them. Thinking that your analysis is correct because it got the results you wanted is just wishful thinking.

The caveat to the above is that the mechanistic procedures suggested by certain quantitatively weak textbooks in quantitatively weak fields (as a psychologist, I ought to know) where you do significance tests and then make further analytic decisions on that basis (e.g., a lack of significant departure from normality means that ANOVA is appropriate) have, to my knowledge, no good basis in mathematics or in science. They are cargo-cult statistics.

$\endgroup$
0
$\begingroup$

The thing to avoid is interpreting a finding of one simple effect being significant and the other insignificant as evidence that the simple effects differ. Also, be careful of multiple tests increasing the Type I error rate.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.