2
$\begingroup$

Suppose I have a linear mixed model with three binary factors A, B, C.

The main effects are all significant but only the interaction A*B is significant, none of the interactions including C is significant.

I then do post-hoc pairwise t-tests with Bonferroni correction. These pairwise tests reveal that the influence of C is only significant for one out of the four possible values of (A,B). For all other values of (A,B) the post-hoc test is clearly insignificant (even without Bonferroni correction).

What does that mean?

Improve this question
$\endgroup$
5
  • $\begingroup$ Any explanation for downvote? The question appears ok to me. I upvote if only to counter-balance the downvote. $\endgroup$
    – amoeba
    Aug 1, 2017 at 11:49
  • $\begingroup$ Thanks amoeba. I was also just wondering why the question was down voted. The question refers to a genuine problem that I have and I also checked the standard statistic literature and couldn't find an answer. $\endgroup$
    – carsten
    Aug 1, 2017 at 11:50
  • 1
    $\begingroup$ FWIW, I don't think this will be answerable unless you show the summary output of your mixed model and post-hoc tests and/or post the data. $\endgroup$
    – amoeba
    Aug 1, 2017 at 11:51
  • $\begingroup$ fair enough, that's possible. i hoped for some general advise on how to deal with the situation of having non significant interaction term but significant post-hoc tests. or some reference to the literature. $\endgroup$
    – carsten
    Aug 1, 2017 at 11:53
  • $\begingroup$ "Significant" is an arbitrary threshold, so if you have p=0.51 in one case and p=0.49 in another, there is no real inconsistency. Apart from that, different procedures have different assumptions and pairwise t-tests might not be equivalent to your mixed model analysis for various possible reasons. Showing model output and/or sharing data is the standard practice on this forum. $\endgroup$
    – amoeba
    Aug 1, 2017 at 11:56

1 Answer 1

Reset to default
1
$\begingroup$

Having a significant AxB interaction term implies that the effect of A changes with a corresponding change in the value of B, and vice versa. This is to say nothing of C, other than that by including it in the model, you are partitioning variance unique to C out of your error term. It is quite possible to have a significant two-way interaction between A and B without C influencing that relationship. One way to look at significant interaction terms is to view them as "permission" to probe further to look for conditional effects; finding a significant AxB interaction suggests that follow-up comparisons ought to be done for A at different levels of B, and vice versa. The finding that the effect of C is significant in one of your comparisons would usually be ignored, given the interaction term is not significant (although, as another user pointed out, significance is an arbitrary threshold, so p-values of .06 and .05 don't really represent a categorical difference in interpretation). Furthermore, it also possible to find a significant AxB interaction, and then, upon probing for conditional effects, you find no significant comparisons. The significant interaction is telling you "the effect of A changes significantly as values of B change", but says nothing about the effect of A at any one particular value of B.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.