Let's say you have a response variable and an independent variable. Your data is measured across several levels of a categorical independent variable. One approach in analysing these data would be to use linear regression to estimate a slope at each level of the categorical independent variable. This is the approach I've used here, using sleepstudy dataset from the R lme4 package (I've stored the betas from each model in lmBetas):

library(lme4); library(plyr); library(ggplot2)
lmBetas <- daply(sleepstudy, .(Subject), function(x) coef(lm(Reaction ~ Days, data=x))["Days"])

Another approach in analysing these data would be to use a mixed effects model to estimate slopes for each level of the categorical independent variable, which in this case is Subject. This is the approach I've taken here (I've stored the betas from the model in lmerBetas):

lmerBetas <- coef(lmer(Reaction ~ Days + (Days | Subject), data=sleepstudy))$Subject[,"Days"]

I have learned that a single mixed effects model, as implemented through the lmer function in R, is more accurate at estimating slopes than a multiple linear regression model applied to multilevel data. This can be demonstrated with this plot of betas from the above models.

betas <- data.frame(method.betas = c(lmerBetas, lmBetas))
betas$method <- c(rep("lmer", 18), rep("lm", 18))

ggplot(betas, aes(method.betas)) + 
  geom_histogram() +
  facet_grid(method ~ .)

The top histogram shows betas estimated using linear regression, and the bottom histogram shows betas estimated using mixed effects. You can see betas estimated using linear regression are more widely spread than those estimated through the mixed effects model.

enter image description here

So finally, my questions:

  1. Is a mixed effects model's higher accuracy in betas estimation connected with the fact that it models intercepts and slopes for each level of the categorical independent variable under a joint probability distribution?

  2. Generally speaking, why is a mixed effects model more accurate in its betas estimation?


The graph you have shown does not show that the coefficients in a mixed effects model are more accurate. It just shows that they are closer together in a mixed effects model. This would mean that the estimate is more precise when it would have been some approximation of the sampling distribution of a single fixed parameter, but that is not what is being shown. When you estimate a mixed effects model, you think that the effects differ across groups and you estimate these different effects. It is a well known property of the mixed effects estimator that the individual effects are "shrunken" towards the overall mean. This is a strength and a weakness; it is how one can get reasonably stable estimates even if the number of observations in each subgroup is small, but it is also a source of bias.

You can read more about this in any textbook on multilevel analysis; I like Tom Snijders and Roel Bosker (1999) Multilevel Analysis. An introduction to basic and advanced multilevel modeling. London: Sage.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.