2
$\begingroup$

I have data which appears to contain a 3 way interaction (the interaction of treatment with time differs across sex) when graphed. However, when I conduct the mixed factor ANOVA the 3 way interaction does not approach significance. Because my interest is whether the treatment X time interaction differs by sex I held sex constant and performed 2 ANOVA on the treatment X time relationships. In this case there was an interaction for females, but not for males.

My question is, why wouldn't the 3 way interaction be significant if the 2 way is significant at one level of the third variable, but not at the other level of the third variable?

More importantly, is the literature to justify these subsequent analyses despite the absence of a 3 way interaction?

Thanks for your insights!!!

| cite | improve this question | | | | |
$\endgroup$
1
$\begingroup$

What you're asking is tantamount to asking, 'why are my two means not significantly different in my t-tests?" Either your data is too variable, or your effect is too small, or the effect does not really exist, or you just don't have enough power.

The difference between significant and not significant can be infinitely small since it's the mere crossing of an exact point. Therefore, it's a meaningless finding that one passes the test and the the other does not. Both interactions could be non significant, or they could both be significant, and still significantly different from each other, or not. The individual tests passing is pretty much completely irrelevant information.

| cite | improve this answer | | | | |
$\endgroup$
  • $\begingroup$ John - Partial ETA squared for the treatment X time effect is 0.17 for females but 0.03 for males. As you would expect with those effect sizes, power is 0.9 for the analysis of females, but .2 for males. Seems to me that the 3-way would be significant. The real issue is that the effect in females is fairly big and seems real. Maybe, maybe not for the males. I would like to make the case that the sexes are different but others are insisting that the 3-way must be significant in order to discuss the female versus male 2 way differences. To me this is getting hung up on arbitrary p values. $\endgroup$ – Docpot13 Feb 20 '16 at 2:07
  • $\begingroup$ Correction, not p values, arbitrary alphas. And actually, it is not "tantamount" to asking why my t-test isn't significant. A more appropriate analogy would be "Why are all of my planned comparisons significant but the omnibus ANOVA isn't?" $\endgroup$ – Docpot13 Feb 20 '16 at 2:20
  • $\begingroup$ In short, your colleagues are correct. This comment adds a lot of info that's not in your question. Comments on an answer are for clarification, not to essentially treat your question as a starting point in a long discussion. Perhaps ask a question now that involves how to discuss meaningful differences and where you see the arbitrary p-value problem appearing. In that question include all the necessary information for someone to really help you. (And, BTW, you probably should have been correcting your p-values for multiple comparisons since you've run about 15 significance tests already.) $\endgroup$ – John Feb 20 '16 at 2:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.