0
$\begingroup$

This is a beginner level question, and I apologise in advance if the answer is obvious.

If a by-subject random slope for a predictor "X" improves the model fit*, but the predictor "X" is not significant (with and without the by-subject random slope for predictor "X"), is there any point in adding the random slope for predictor "X"? The reason why I am asking this is because I always thought that the point of adding random slopes was to minimise the risk of Type I error, but there is no risk of Type I error with regards to predictor "X" if it is not significant in the first place.

*as judged by the pairwise comparisons using the LRT

Here is an illustration of what I mean:

ordinal_Eiii <- clmm(Understanding ~ zAge + EnglishProficiency + zSTOFHLA+Education + zFRES + zRDL2 + (1|Text) + (1|Subject), data = Study2.pairs, link = "probit", threshold = "flexible")

In the above model:

                               Estimate Std. Error z value Pr(>|z|)
zRDL2                          -0.02217    0.13923  -0.159  0.87350 

After the addition of a random slope:

ordinal_Eii <- clmm(Understanding ~ zAge + EnglishProficiency + zSTOFHLA+Education + zFRES + zRDL2 + (1|Text) + (zRDL2 + 1|Subject), data = Study2.pairs, link = "probit", threshold = "flexible")

                               Estimate Std. Error z value Pr(>|z|)
zRDL2                          -0.18864    0.18454  -1.022  0.30669  

Pairwise comparisons using the LRT:

         no.par    AIC  logLik LR.stat df Pr(>Chisq)   

ordinal_Eiii 19 1440.1 -701.06
ordinal_Eii 21 1430.6 -694.31 13.503 2 0.001169 **

In addition to the above, the inclusion of the random slope for zRDL2 changes the estimates of the other predictors. Thus, is the inclusion of random slope for zRDL2 warranted since it decreases the risk of committing a Type I error with regards to the remaining predictors? Am I understanding this right?

$\endgroup$
1
$\begingroup$

I'm a little bit confused by your use of "pairwise comparisons" to refer to a nested model comparison using the likelihood ratio test.

The likelihood ratio is testing to see if: (a) the variance of the slopes is greater than zero, or (b) the covariance between the random slopes and intercepts is different from zero.

I would first adjust your p-value for the fact that it is testing one hypothesis bounded at zero (i.e., variances cannot dip below zero).

What you have is a significant slope. The fact that this changes the fixed estimate part of zRDL2 doesn't matter much. When I run mixed/multilevel/hierarchical models, I do a top-down approach: I start with a saturated random-effects structure. Then, I start picking away at the random effects and doing nested model comparisons along the way. If there is no significant change when I take something out, then I leave it out.

Then I move onto the fixed effects, doing the same thing.

I think your model with the random slope is an appropriate model, given what you have presented. To the best of my knowledge, this is not because it decreases the risk of committing a Type I error, but because it is the best way to model the variance in your dependent variable. I find that graphing random slopes or showing a spaghetti plot is very, very telling in how the random effects structure looks.

EDIT:

From the comment below:

What I am wondering is: does it make sense to add a random slope of a variable, if that variable is not significant in the first place?

Yes, this makes sense. What it tells you is that the average slope does not differ significantly from zero (i.e., the coefficient's p-value > $\alpha$); however, there is significant variance in slopes among people (i.e., the random slope is statistically significant). What would this look like?Random slopes and intercepts

The thick red line represents the average slope: It is not significantly different from zero. The thinner black lines represent the slopes for each person. Note that these vary wildly from positive to negative: The variance is significantly different from zero. This is like what you have in your model.

What is interesting here is that, if you were interested, you could look at moderators at the individual level that might explain when someone has a positive or negative slope.

$\endgroup$
  • $\begingroup$ Hi Mark. Thank you for your answer and for the useful link on how to adjust my p-value. All my fixed effects are variables of theoretical interest, so a finding that one is not significant is a finding in itself. I started from the fixed-effects, and then I moved onto the random effects. What I am wondering is: does it make sense to add a random slope of a variable, if that variable is not significant in the first place? From your answer it seems that the answer to my question is yes, as long as the random slope improves the model fit -- is my interpretation of your answer correct? Thanks. $\endgroup$ – user139190 May 6 '17 at 10:01
  • $\begingroup$ I wanted to include a picture, so I replied above by editing my original answer. $\endgroup$ – Mark White May 6 '17 at 12:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy