I'm reviewing a paper in which there are two independent variables (A and B), each with two levels (A1 and A2; B1 and B2). There is a significant two-way interaction. The authors ran simple effects testing both ways, first each level of A between levels of B, then each level of B between levels of A.
The first test makes sense: A1 is significant differently between B1 and B2; A2 is NOT significantly different between B1 and B2. It is easy to account for--and explain--the significant interaction.
The second test doesn't make sense: B1 is significant differently between A1 and A2; B2 is ALSO significantly different between A1 and A2. Both are significant, which is counter-intuitive to what an interaction means.
Could anyone explain this a little better? I don't have access to the standard deviations or distribution measures related to the original data, but I am thinking that there is a violation of the ANOVA assumption related to normal distributions. Any alternative reasons for why the numbers are doing what they are doing?