Including (and interpreting!) random intercepts and/or slopes in linear mixed models

Question

I'm new to linear mixed modeling, and have some theory-driven questions that I'm not sure how to analytically resolve.

I am analyzing experimental data with a within-subjects factor (discount). My theory hypothesizes that the effect of this within-subjects factor is contingent upon a between-subjects characteristic of respondents (iipm). Because my data is in long form, and respondents are making 8 choices over time, I model my data as follows:

library(lme4)
m1 <- glmer(chose ~ iipm*discount + product + (1|id) + (1|time), data=long1, 
        family="binomial")

All that I'm trying to do here is fit a simple model that accounts for the dependence between observations for a single subject (1|id), and the potential effect of making several choices in a row (1|time).

However, my theory further specifies that this relationship should not be affected by the inclusion of other demographic variables in the model (let's say ideology and partisanship). So, based on some reading I've done (as well as previous answers on this site), I fit the following model:

m2 <- glmer(chose ~ iipm*discount + product + (1 |id) + (1|time) + (1|partisanship) +
       (1|ideology) , data=long1, family="binomial")

Because random slopes goes beyond my expertise, I'm just using random intercepts to see what happens when I account for baseline variation amongst individuals attributable to their partisanship and ideology. However, if I were to use random slopes to essentially say that the effects of partisanship and ideology vary on an individual basis, even after accounting for baseline variability, I should specify the following model:

m3 <- glmer(chose ~ iipm*discount + product + (1 + partisanship +ideology |id)  +
  (1|time) , data=long1, family="binomial")

To test the hypothesis that this baseline variability doesn't matter, I then run a likelihood ratio test comparing the two models:

library(lmtest)
lrtest(m2,m1) # p=.349
lrtest(m3,m1) # p=.416

If there's no improvement in fit (p>.05), I (very tentatively) interpret this as support for my hypothesis that demographics and ideology don't matter.

Is this a right way to approach the data, or is there a more sophisticated way to test this hypothesis using multilevel modeling? Any expertise and advice is greatly appreciated.

Answer 1

I think you're specifying a great deal of random effects which are better modeled as fixed effects since you'll directly estimate the odds ratio for their association with outcome. For instance, the question of whether partisanship modifies the relationship of discount and product is simply done as a test of interaction:

m1 <- glmer(chose ~ partisanship + iipm*discount + product + (1|id) + (1|time), data=long1, 
    family="binomial")
m2 <- glmer(chose ~ partisanship*(iipm*discount + product) + (1|id) + (1|time), data=long1, 
    family="binomial")

lrtest(m2, m1)

Under the null hypothesis, the interaction effects have a log odds ratio of 0. Note we have a LOT of such effects: partisianship:iipm, partisianship:discount, partisianship:product and the three way effect partisianship:iipm:discount. Another important modification to the null model is that I included partisanship as a fixed effect in the null model. This is because when testing for effect modification, you need to ensure that the interaction parameters have the interpretation as a difference in odds ratios in the main effect for a unit difference in the modifier. I think "Regression Methods in Biostatistics" from Vittinghoff, ch. 7 addresses this.

While the meaning of the p-value in the interaction analysis is difficult to interpret, I guarantee you it's much easier than the test of random effect you included above. Worse, the variance component test doesn't address the question you stated.

Lastly,

If there's no improvement in fit (p>.05), I (very tentatively) interpret this as support for my hypothesis that demographics and ideology don't matter.

This is not the correct interpretation of a $p$-value. Basic NHST says there was insufficient evidence to suggest otherwise. Given the complexity of this model, I'd be very dubious of the interpretation you provided.