0
$\begingroup$

I'm reviewing a paper in which there are two independent variables (A and B), each with two levels (A1 and A2; B1 and B2). There is a significant two-way interaction. The authors ran simple effects testing both ways, first each level of A between levels of B, then each level of B between levels of A.

The first test makes sense: A1 is significant differently between B1 and B2; A2 is NOT significantly different between B1 and B2. It is easy to account for--and explain--the significant interaction.

The second test doesn't make sense: B1 is significant differently between A1 and A2; B2 is ALSO significantly different between A1 and A2. Both are significant, which is counter-intuitive to what an interaction means.

Could anyone explain this a little better? I don't have access to the standard deviations or distribution measures related to the original data, but I am thinking that there is a violation of the ANOVA assumption related to normal distributions. Any alternative reasons for why the numbers are doing what they are doing?

Thanks.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Perhaps you could tell us what you think an interaction means. That would give us a point of departure for explaining this phenomenon. $\endgroup$
    – whuber ♦
    Commented Jan 12, 2018 at 20:58
  • $\begingroup$ I don't want to violate the confidentiality of the review process. The meaning of an interaction is the same regardless of the independent variables. In this case, the interaction indicates that there is a significant difference between the two levels of A, but only at one level of B. That is a traditional definition of an interaction. In the case I described, the alternative analysis reveals a significant difference between the two levels of B, but at both levels of A. $\endgroup$
    – DouglasParham
    Commented Jan 13, 2018 at 3:14
  • $\begingroup$ I do not quite understand the problem: it could be A1=1, A2=-1, B1=2, B2=-2, A1B1=10, A1B2=-10, A2B1=-10, A2B2=10 with all interactions being highly significant. Only if e.g. interactions A1B1, A1B2 were both of the same sign one could tell there is something wrong. $\endgroup$
    – F. Tusell
    Commented Jan 13, 2018 at 12:51
  • $\begingroup$ I think you may be employing an overly (and unusually) narrow understanding of "interaction." First, the standard meaning makes no reference to "significance"--that's a separate issue. Second, an interaction typically means that the way in which $B$ is associated with the response varies with levels of $A$. Your description of the alternative analysis sounds like a classic description of a significant interaction. $\endgroup$
    – whuber ♦
    Commented Jan 16, 2018 at 16:00

1 Answer 1

Reset to default
1
$\begingroup$

There is nothing counter-intuitive here if you have the right definition of an interaction. It definitely is not that (to quote your comment):

In this case, the interaction indicates that there is a significant difference between the two levels of A, but only at one level of B.

First, judging significance as a "yes/no" phenomenon is an error here. Even if the relation of A to the DV are in opposite directions at different levels of B, there could be any combination of significances for A, B and the interaction.

Second, it could be that the relation of A to the DV could be in the same direction for B1 and B2 and both be significant, but much more extreme at one level of B than the other.

There are other possibilities too, but those are enough to show how what you say could happen.

An interaction simply means that the relation of one IV to the DV are different at different levels of the other IV.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you very much, Peter. I think I understand the logic. I just am struggling to grasp that the simple effects testing for a significant interaction did not identify significant and non-significant differences among levels. In the case I described, B1 is significant differently higher at A1 than at A2; B2 is ALSO significantly higher at A1 than at A2. Thus, both groups (B1 and B2) are significantly higher at A1 than at A2. To my mind, this simply seems to be replicating--although not really--the main effects test for A (where we would collapse B and compare A1 vs. A2). $\endgroup$
    – DouglasParham
    Commented Jan 14, 2018 at 21:15
  • 1
    $\begingroup$ Whether it did or not is not really relevant to whether there was an interaction. Don't look at significance, look at effect sizes. They will be different. $\endgroup$
    – Peter Flom
    Commented Jan 15, 2018 at 12:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.