Should a within-subjects variable be modeled with a random intercept and slope even if within-subjects correlations are minimal?

Question

I am evaluating an analysis of an experiment in which each participant was shown 5 pairs of stimuli which represented options that participants could choose between - call the two options in each pair Choice 0 and Choice 1. Each participant chose either Choice 0 or Choice 1 (which were presented in random order) for each stimulus. In addition, there were also two between-subjects conditions:

• For a given participant, either Choice 0 always had Property A (and Choice 1 did not), or Choice 1 always had Property A (and Choice 0 did not).
• For a given participant, either Choice 0 always had Property B (and Choice 1 did not), or Choice 1 always had Property B (and Choice 0 did not).

The researchers are trying to predict participants' five choices (0 or 1 in each case) using the following logistic regression model:

Choice ~ Intercept + Stimulus + PropertyA + PropertyB + PropertyA*PropertyB

• 'Stimulus' has 5 levels corresponding to the 5 pairs of stimuli and is dummy/treatment-coded, they have arbitrarily chosen the first level as the reference level.
• PropertyA is coded 1 if Choice 1 had Property A, 0 otherwise.
• PropertyB is coded 1 if Choice 1 had Property B, 0 otherwise.

My initial impulse is to suggest that they should re-run the analysis using effect coding for Stimulus since they have no reason for any particular value of this factor to serve as the reference level, and that furthermore, because each participant is making 5 choices which are likely correlated, it would be more appropriate to use a mixed effects model with a random intercept and random slope for Stimulus, e.g.

Choice ~ Intercept + (1 + Stimulus | ID) + PropertyA + PropertyB + PropertyA*PropertyB

However, given the particular details of this experiment it is possible that any given participant's responses on the five stimuli that they are presented with were only weakly correlated. If so, then is it still important for them to run this as a mixed effects model? For example, if Stimulus has a VIF of, say, less than 3 in their current model, should I still insist that they re-run it with a random intercept and random slope for Stimulus?

Any other important critiques (of either the researcher's analysis or of my planned response to it) would be welcome.

Answer 1

I think you are on the right track with your suggestions in terms of using a mixed effects model. The mixed effects model will provide an estimate of the variance in the outcome at the ID level. If it is minimal, then you might consider abandoning the mixed model, but then again, many would argue that unless that variance is 0, it is important to utilize a mixed model because it better matches how the data was generated.

The one thing to keep in mind is that if stimulus is indeed a factor variable, then to properly model it as a random slope, you will need to create 0/1 indicators for all levels (minus the holdout), and then each of the 0/1 indicators will need to be included as random slopes. I believe that lme4 will do this automatically if Stimulus is coded as a factor variable. However, this may be a difficult model to fit, depending on how much data you have. An alternative that doesn't require all those slopes and covariances would be to switch to the following:

m3 <- lmer(Choice ~ Intercept + (1 | ID) + (1 | Stimulus:ID) + PropertyA + PropertyB + PropertyA*PropertyB, dat)

As nicely described here, this fits a model where the effect on the outcome is allowed to vary both by ID and then also uniquely for each combination of Stimulus and ID. It avoids the complicated variance-covariance matrix involved in fitting separate slopes for each Stimulus.