# Exploring a two-way interaction in multi-level regression, why do I get significant contrasts when the confidence intervals overlap on the graph?

I'm exploring a significant two-way interaction in a multi-level random-effects regression. The graph (below) appears to show the interaction is driven by differences at low X, and that at high X the moderator (V2) doesn't matter. Is there a reason why I get highly significant contrasts at high X when the confidence intervals appear to overlap?

Interaction graph (circles show where the reported contrast is being conducted):

Contrast test at high X (circled on the graph):

      Y   |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
----------+-------------------------------------------------------------------
z_X = 1.5 |  -.1288761   .0471769    -2.73   0.006    -.2213412   -.0364111
------------------------------------------------------------------------------


That's surprising. Is this something to do with multi-level (nested/panel) data? I'm not very familiar with inferential stats using multiple levels simultaneously. I would have expected that if the 95% confidence intervals overlap, as they clearly do for z_X = 1.5, that the contrast there would be not significant or close to that. Thanks for reading.

The following are relevant posts, but I sadly admit I did not follow the explanations: Relation between confidence interval and testing statistical hypothesis for t-test

Large overlap between confidence intervals, although z test for difference was significant

I'm also new around here, so please let me know if I could improve how I asked this question.

• I've added the image to the body of the question for you (which you'll be able to do once you've got sufficient reputation) Apr 16, 2015 at 22:53
• You say "I would have expected that if the 95% confidence intervals overlap, as they clearly do for z_X = 1.5..." -- are you basing this on the confidence bands overlapping in the graph, or on some other information? Apr 16, 2015 at 22:54
• The existence of overlap at some given (high) x-value isn't really related to whether there's a significant interaction. Indeed, its not clear to me why the picture should lead to any surprise -- it's entirely consistent with a significant interaction. You could also see overlap near the ends of the range when there was just a significant main effect. Apr 16, 2015 at 23:52
• I'm sorry, I appear to have been a bit unclear. The interaction is significant, small effect size standardized b = .03. Then I tested for what I call above "contrasts," that is, tests of equivalence for values of Y at high vs. low V2 at low X, highly significant; then values of Y at high vs. low V2 at high X, and that's the stat I quoted initially, and based on visual inspection of the graph's CIs I didn't think it would be significant but it was. I don't think the question has been answered yet. Thanks James for image posting! Apr 21, 2015 at 20:59
• Overlapping CIs of two estimates does not necessarily mean the difference of the estimates is not significant. This is just another example where that happened to be the case. Apr 21, 2015 at 21:08