2x2 ANOVA, no main/interaction effects, post-hoc cell mean comparisons necessary/allowed?

Question

I have a 2x2 between-subject design with unequal cell sizes. I ran an ANOVA with type I sums of squares to account for the unequal cell sizes.

Neither of the two main effects nor the interaction effects were significant (all ps > 0.1).

With an overall of 2x2 = 4 cells/groups there are obviously 6 possible pairwise comparisons, i.e. 6 individual differences between cell means that might be statistically significant. Can I conclude from the non-significant main and interaction effects that none of these pairwise comparisons are statistically significant?

As the plot suggests it would be particularly interesting to examine the difference between the two means on the left-hand side, i.e gain-absent vs. gain-present.

Would it be legitimate/necessary to run a Tukey-Kramer test (due to unequal cell sizes) to test this? If so, how do I do this in R? (Here I'm assuming that the standard TukeyHSD() function would be invalid due to unequal cell sizes)

Answer 1

No, you cannot conclude that none of the comparisons are significant. If the figure represents your data then I'm guessing that you're going after present higher than absent in gain but not in loss. If that's the case be very cautious because that's exactly what your interaction tested. And, a non significant interaction already told you that the difference between significant and not significant was not, itself, significant.

You might even only find gain:present higher than loss:absent, but what would that mean? Are all of tests even sensible or interpretable?

In short, before venturing forth with more tests seriously consider what they mean. I suggest that if you really consider the ANOVA result it has already tested any meaningful questions you'll have about the data. If you decide to make the comparisons anyway then answers to this question address yours as well.