1
$\begingroup$

I'm in the process of building my mixed models, and unfortunately I encountered a problem when creating the random effects structure. I have two random effects: ResponseId (i.e., participant number) and stim_Id (i.e., item number). I have added random intercepts for both effects, since they are both source of non-independence in my data. I have also added two by-participant random slopes, for each of my two within-subject variables, here named face_type and stim_gender, both being two-level factors. Unfortunately, if I do add both of the random slopes, the model fails to converge. I have tried some ways to solve it but I think the rigth thing to do is to get rid of one of the random slopes.

I have looked at the output of my model (model and output showed below) and I can see that the intercept for ResponseId is 0 or close to 0, which means it doesn't add much to the overall model. But, does it mean that I should get rid of the random intercept here? I would find it unusual to have a model with fixed intercept and random slope. I'd rather expect to remove the random slope and keep the random intercept, however the 0 value of intercept (while the value of slope seems to be 0.04) somewhat confuses me. Does anyone have experience with this?

Important note:

  • After fitting the fixed effects, only the predictor of face_type is significant, no matter how I structure the random effects. I suppose that would in itself predict that random slope with stim_gender doesn't add much, is that correct? Even if that's true, I don't want to use this backwards approach as justification to get rid of one variable of another...

My model and the output:

lmer(rating ~ 1 + (1|stim_Id) + (1 + face_type|ResponseId) + (1 + stim_gender|ResponseId), REML=F, data=df)



 Linear mixed model fit by maximum likelihood . t-tests useSatterthwaite's method [lmerModLmerTest]

 AIC      BIC   logLik deviance df.resid 


13883.8  13942.5  -6932.9  13865.8     5047 

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-4.8467 -0.3878 -0.0989  0.1336  6.1131 

Random effects:
 Groups       Name            Variance Std.Dev. Corr 
 ResponseId   (Intercept)     0.24997  0.5000        
              face_typereal   0.09006  0.3001   -1.00
 ResponseId.1 (Intercept)     0.00000  0.0000        
              stim_gendermale 0.03962  0.1990    NaN 
 stim_Id      (Intercept)     0.53824  0.7336        
 Residual                     0.82318  0.9073        
Number of obs: 5056, groups:  ResponseId, 79; stim_Id, 64

Fixed effects:
            Estimate Std. Error      df t value  Pr(>|t|)    
(Intercept)   0.4592     0.1010 85.8841   4.547 0.0000177 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
optimizer (nloptwrap) convergence code: 0 (OK)
boundary (singular) fit: see ?isSingular
Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

A few points to note:

  1. The model has converged. However it has converged to a singular fit which usually means the random structure is over-fitted.

  2. You specify face_type and stim_gender as random slopes, yet, you do not fit either as fixed effects. This means that you implicitly want the mean "effect" of both to be zero. This is not usually what you want, so I would suggest fitting them as fixed effects as well:

lmer(rating ~ 1 + stim_gender + face_type + (1|stim_Id) + (1 + face_type|ResponseId) + (1 + stim_gender|ResponseId), REML=F, data=df)

  1. Do face_type and stim_gender both vary within ResponseId ? If not, then there is no point trying to fit random slopes.

  2. Think about whether you really do need random slopes in this model. Does the underlying theory support this ?

  3. Assuming that the answer to 3. and 4. is affirmative and the model I suggested in 2. is still singular, then you need to simplify the random structure. Quite often this can be done by removing all random slopes, and if that model converges, then trying adding one back at a time. Alternativelty it might be that there is very little variation within one or more of the grouping variables ResponseId and stim_Id. A principled way to investigate this is described in these two answers:
    Should I remove random intercepts from my model?
    How to simplify a singular random structure when reported correlations are not near +1/-1

Improve this answer
$\endgroup$
7
  • $\begingroup$ Hi Robert, thank you for the response. What you're saying is very interesting because it's contrasting the guidelines I followed when building the models, specifically, I read that the first thing to do is to create maximal random effects structure. That is why I have started by creating random intercepts/slopes without adding the predictors. The fit of the model is also improved according to AIC/BIC as well as Likelihood Ratio Test when by-subject random slopes are introduced. Do you suggest that despite that I shouldn't include them unless the underlying theory supports this? $\endgroup$
    – Ola_88
    May 16, 2021 at 10:23
  • 1
    $\begingroup$ @Ola_88 the advice in the paper you refer to, which I assume is the one by Barr et al is extremly bad and leads to many many many qustions like yours on this site. Take a look at this answer for more info: stats.stackexchange.com/questions/482307/… Also note that Bates is the primary author of lme4::lmer. Also you mustn't be guided by black and white "rules" like AIC and LRTs. As for including random slopes, you should only do so when supported by theory AND the data. $\endgroup$
    – Robert Long
    May 16, 2021 at 10:25
  • $\begingroup$ What I forgot to add is, you are correct in 1. - I checked the model and indeed the random structure is over-fitted, which I dealt with by removing the by-subject random slope for stim_gender. I must say though, there's little reasoning behind this decision, other than the fact that out of the two, face_type has larger variance within the ResponseId variable. That is, I rely on the idea of maximising the model's fit here rather than theory... $\endgroup$
    – Ola_88
    May 16, 2021 at 10:35
  • $\begingroup$ okay I see, I was indeed referencing the Barr et al. paper... I shall follow your advice then, thank you. Seems the enxt thing to do for me is to try to better understand the idea behind random slopes, as I'm not entirely sure in what way should theory can drive the decision to include or not include the random slopes. $\endgroup$
    – Ola_88
    May 16, 2021 at 10:41
  • $\begingroup$ @Ola_88 well the problem with maximising model fit by increasing model complexity is that it very often leads to an overfitted model. It's essentially the same a fitting a 9th order polynomial to a dataset of 10 observations - you will definitely get an excellent fit, but the inferences will be nonsense and the predictive ability with an out-of-sample dataset will be completely rubbish. You seem to have quite a large dataset so one thing you could do is hold back some of your data and test your model on that hold-out sample. $\endgroup$
    – Robert Long
    May 16, 2021 at 10:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.