Significant cross-level interaction despite lack of variance in level-1 slopes

Question

I have a logistic HLM model with one level-1 predictor and without level-2 predictors. Random variance components are significant for intercepts, but far from significant (p>.5) for slopes. In my understanding this means that there is some variation in the level-1 intercepts but no or very little variation in the level-1 slopes. Hence, if I add a level-2 predictor to the model, this predictor could predict level-1 intercepts but not level-1 slopes (because there is no variation to predict, right?).

However, if I add a level-2 predictor, I find that both the intercepts and the slopes are significantly predicted. The cross-level interaction looks very nice and is very much in line with my hypotheses. But how can there be a significant cross-level interaction with no varation in level-1 slopes? How can I interpret this result?

In this related thread (which was very satisfyingly answered) I cannot find an answer to my question: Can I probe cross-level interactions without random slope in a hierarchical linear model?

Answer 1

The cross-level interaction is possible when the variance of random slopes is (close to) zero. In fact it never is exactly zero, and one decides whether it is a zero choosing some criteria, depending on which you may come to different conclusions. Wald test, likelihood ratio test and bootstrapped confidence intervals are most used criteria to decide whether variance of random slopes is actually zero. They may come out with different results.

Random slopes in logistic HLM are assumed to be continuous, they play as a dependent variable in cross-level interactions. Therefore, your question may be reduced to a more simple one: is it possible to get significant effect when the variance of the dependent variable (slopes in your case) is very small. It is possible, because significance of the effect is higher if the sample size is larger, variance of independent variable is larger and residual is smaller.

Consider this example. The variance of y is 0.00000895 which some tests would identify as zero. At the same time, the effect of x is highly significant (mostly because the residual variance is also really small).

> x <- runif(100, 0, 1)
> e <- rnorm(n=100, mean=1, sd=0.001)
> y <- 0.01*x + e
> var(y)
[1] 0.00000895
> 
> model<- lm(y ~ x)
> summary(model)

Call:
lm(formula = y ~ x)

Residuals:
        Min          1Q      Median          3Q         Max 
-0.00268828 -0.00068731 -0.00004948  0.00072524  0.00187309 

Coefficients:
             Estimate Std. Error t value            Pr(>|t|)    
(Intercept) 1.0001458  0.0002009 4978.37 <0.0000000000000002 ***
x           0.0097976  0.0003340   29.34 <0.0000000000000002 ***

Going back to your original question, the point is that this kind of significant cross-level interaction has no or little meaning as it explains a negligibly small proportion of variance in your dependent variable.