Let's say I have a random effect intercept. For example:

lme4::lmer(yield ~ 1 + (1|Batch))

How is that different than just ordinary regression using regularization or a prior?

 arm::bayesglm(yield ~ 1 + Batch,...)

They seem roughly equivalent to me.

Addendum

Unified view on shrinkage: what is the relation (if any) between Stein's paradox, ridge regression, and random effects in mixed models? asks something similar. The accepted answer claims that they are equivalent.

However, I am not convinced that it is correct, otherwise what is the need for a package lme4, which is typically more computationally expensive to fit?

My intuition is that regularization is equivalent to a mixed model with a diagonal covariance matrix for the random effects. Whereas something like lme4 supports more general covariance structures.

  • 2
  • @amoeba I am not convinced that the answer there is correct. See the addendum to my question. – Jeff Aug 18 at 18:06
  • 1
    part of the difference is that mixed models are simultaneously estimating the appropriate degree of shrinkage (under the assumption that the random effects are Normally distributed). e.g. for bayesglm you have to specify the prior on the fixed effects. – Ben Bolker Aug 18 at 21:17
  • 1
    I think the answer there is correct, but of course it does not imply that the whole lme4 is just equivalent to ridge regression. lme4 can estimate very complicated covariance structures; but your example with one random intercept term is indeed very similar to having a L2 penalty (or prior), as explained in the linked answer. It's true what @BenBolker wrote though; it's more like empirical Bayes, where the prior is estimated from the data (or like ridge regression when the penalty is optimized over). – amoeba Aug 18 at 21:58
  • My question was more broader. I guess the example I provided is too simple. I was trying to get at the heart of what could be modeled with a mixed model that can't be accomplished with simpler approaches, e.g., bayesglm. For me this has practical considerations, as I am looking at modeling things that are possibly too expensive with lme4. – Jeff Aug 18 at 22:02

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.