Interpret contradicting output of lmer model with categorical interaction in R

Question

I am struggling to interpret my output in R. It does not make sense to me.

I first regressed participants' ratings (= value) on manipulations (= variable: H high, AI high, H low, AI low, and control). I then checked whether this effect is moderated by the participants' experience (= Frequency :7 groups, the higher, the less experience). Frequency was added as an interaction factor in my model:

H4_mod <- lmer(value ~ variable * Frequency + (1|subject), data = 
  ANALYSIS_long, REML = FALSE)

Then, in order to find out whether the interaction effect is statistically significant, I compared the H4_mod with a reduced model:

red_H4_mod <- lmer(value ~ variable + Frequency + (1|subject), data = 
  ANALYSIS_long, REML = FALSE)

An ANOVA and a look at cAICs revealed that the interaction is statistically significant. So far, so good.

Now, I want to find out what the effect looks like. I predicted that the higher the experience, the lower the effect of the prediction (value ~ variable).

I get confused with the output I get from

summary(H4_mod)

Fixed effects:
                               Estimate Std. Error         df t value Pr(>|t|)    
    (Intercept)               6.846e+00  2.872e-01  8.334e+02  23.839  < 2e-16 ***
    variable[T.2]             9.542e-01  3.223e-01  1.032e+03   2.960 0.003145 ** 
    variable[T.3]            -8.000e-01  3.223e-01  1.032e+03  -2.482 0.013226 *  
    variable[T.4]             1.104e+00  3.223e-01  1.032e+03   3.426 0.000638 ***
    variable[T.5]            -1.154e+00  3.223e-01  1.032e+03  -3.581 0.000359 ***
    Frequency2                3.310e-01  3.739e-01  8.334e+02   0.885 0.376362    
    Frequency3                5.490e-01  3.650e-01  8.334e+02   1.504 0.132893    
    Frequency4               -5.164e-01  5.615e-01  8.334e+02  -0.920 0.358013    
    Frequency5                2.875e-01  6.421e-01  8.334e+02   0.448 0.654468    
    Frequency6                5.417e-02  6.043e-01  8.334e+02   0.090 0.928602    
    Frequency7               -1.156e+00  5.295e-01  8.334e+02  -2.183 0.029329 *  
    variable[T.2]:Frequency2 -3.397e-01  4.197e-01  1.032e+03  -0.809 0.418551    
    variable[T.3]:Frequency2 -4.087e-01  4.197e-01  1.032e+03  -0.974 0.330432    
    variable[T.4]:Frequency2 -5.592e-01  4.197e-01  1.032e+03  -1.332 0.183033    
    variable[T.5]:Frequency2 -2.545e-01  4.197e-01  1.032e+03  -0.606 0.544378    
    variable[T.2]:Frequency3 -5.901e-01  4.097e-01  1.032e+03  -1.440 0.150082    
    variable[T.3]:Frequency3  5.128e-03  4.097e-01  1.032e+03   0.013 0.990015    
    variable[T.4]:Frequency3 -4.837e-01  4.097e-01  1.032e+03  -1.181 0.238045    
    variable[T.5]:Frequency3  2.875e-01  4.097e-01  1.032e+03   0.702 0.482980    
    variable[T.2]:Frequency4  3.407e-02  6.303e-01  1.032e+03   0.054 0.956904    
    variable[T.3]:Frequency4  2.471e-01  6.303e-01  1.032e+03   0.392 0.695154    
    variable[T.4]:Frequency4  2.841e-01  6.303e-01  1.032e+03   0.451 0.652301    
    variable[T.5]:Frequency4  1.025e+00  6.303e-01  1.032e+03   1.626 0.104285    
    variable[T.2]:Frequency5 -5.708e-01  7.208e-01  1.032e+03  -0.792 0.428548    
    variable[T.3]:Frequency5 -2.250e+00  7.208e-01  1.032e+03  -3.122 0.001848 ** 
    variable[T.4]:Frequency5 -1.288e+00  7.208e-01  1.032e+03  -1.786 0.074342 .  
    variable[T.5]:Frequency5 -1.413e+00  7.208e-01  1.032e+03  -1.960 0.050295 .  
    variable[T.2]:Frequency6 -1.970e-01  6.783e-01  1.032e+03  -0.290 0.771525    
    variable[T.3]:Frequency6 -9.714e-01  6.783e-01  1.032e+03  -1.432 0.152417    
    variable[T.4]:Frequency6 -3.327e-01  6.783e-01  1.032e+03  -0.491 0.623863    
    variable[T.5]:Frequency6 -1.203e+00  6.783e-01  1.032e+03  -1.773 0.076448 .  
    variable[T.2]:Frequency7  3.658e-01  5.944e-01  1.032e+03   0.616 0.538350    
    variable[T.3]:Frequency7 -6.000e-02  5.944e-01  1.032e+03  -0.101 0.919610    
    variable[T.4]:Frequency7  2.583e-02  5.944e-01  1.032e+03   0.043 0.965340    
    variable[T.5]:Frequency7  4.742e-01  5.944e-01  1.032e+03   0.798 0.425178    
    ---
    Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Can someone explain it to me? The intercept is my control manipulation ("variable"), but what happened to Frequency 1? I read that only the Frequency 7 group has a significant impact on the relationship (value ~ variable). When I look further down at the (what I think) actual interaction effects, these are only significant for Frequency 5.

Furthermore, I tried to figure out the relationships by looking at an effects plot

e <- allEffects(H4_mod)
plot(e, multiline = TRUE, confint = TRUE, ci.style = "bars")

Unfortunately, this confuses me even more... Can someone help me interpreting the results? So far, I can only say that Frequency significantly interacts with the value ~ variable relationship. But I would love (and have) to make a statement about the direction.

THANK YOU!

Answer 1

This is really quite simple, the only issue is that you've fallen into the trap of categorizing ordinal values simply because they are ordinal. It's not surprising that the the global test of (5-1) * (7-1) = 24 fixed effects representing the high dimensional interaction is statistically significant, the chance for overfitting and small sample bias is high. Even the plot, which in most cases would be an intelligent summary of such high dimensional output (because it shows Frequency continuously), shows massively overlapping CI bounds.

If you want a simple and easy to interpret output, consider fitting the Frequency and Variable measures as numeric instead. Be sure that the numeric value actually corresponds to the observed level. This gives you a more powerful and more interpretable test.