Difficulty interpreting the random effects from a Logistic Mixed Effects Regression

Question

We are using a dataset with ~2,000 households across 20 different sites across the world. We are using a Mixed Effects Logistic Regression to examine the effect of income groups (categorical IV) on food insecurity (binary DV). We include other variables in the models, but this is our principal relationship of interest. We are using site as a grouping variable. Based on our theoretical understanding – we include a random intercept as well as a random slope term, because we expect the effect of income group to vary across sites.

We have produced the fixed effects (e.g., income group, and other relevant variables) as well as the random effects (site and income group). The fixed effect Odds Ratios tell us the “average” effect of IV on D, accounting for other variables in the model. When we plot the random effects as Odds Ratios, we understand this as meaning how the Odds Ratios vary across sites – or what is the effect of IV on D in a specific site. However, we also recognize that this is different than running site-wise regression models. We have several interpretational questions as it relates to the full model (~2000) effect of IV of D, the random effects, and the site-wise models (which we also ran, i.e., 20 logistic regressions):

1. What is the difference between running individual site-specific models (20), and using the random slopes extracted from the full model to investigate site-wise differences? That is, we know the individual site models are different from what we see as the random slopes -- but how are these different?

2. The random slopes for each site are sometimes inconsistent from the full model slope. We are interested in giving attention to the average or full model slope, but not at the expense of overshadowing particular sites if those are very different. Can anyone provide interpretational guidance here?

3. Most examples (we found) that look at individual random slopes and compare this to the ‘full’ model (fixed portion), all point generally in the same direction although their slope might vary. There are of course cases where this is more complicated; some sites could have negative slopes (or <1 OR) that run counter to the ‘full model’ effect. How can these be interpreted? If there is a large variation in random effect values across sites – how important the fixed effects values are?

Answer 1

I am going to provide some general thoughts on your questions and hopefully this will help you to seek more detailed information on these topics. Before doing so, it is not clear to me exactly how you are getting estimates of the random slopes. Is it Empirical Bayes prediction of the means or the modes?

A first point is that you must remember that these predictions have associated uncertainty, which you should be able to get from your software. You cannot think of the predictions as exact point estimates of each country's slope, because they are not that at all. They are the mean or mode of a posterior distribution of slope estimates that are consistent with your model and its assumptions. This is one reason that you see so-called caterpillar plots of random effect estimates because they show you exactly how much uncertainty is baked into the prediction. In most cases, you will see that the majority of groups are not different from one another once you look at the uncertainty intervals.

A second point to remember is that the predicted random effects are all relative to the fixed effect. They are positive or negative deviations off the fixed effect. A value of 0 for a random effect means that the predicted effect is equivalent to the fixed effect.

A third point to remember about random effects is that, because they are given a normal distribution, they tend to be smaller than if you had estimated the slopes for each group separately. This feature of random effects is called by various names, including partial pooling, regularizing, or shrinkage, with the key idea being that the slopes you get from a mixed effects model for individual groups will be pulled toward the fixed effect slope estimate when the group size is small.

Hopefully this information is helpful in considering your results.