28
$\begingroup$

By regularization techniques I'm referring to lasso, ridge regression, elastic net and the like.

Consider a predictive model on health care data containing demographic and diagnosis data where length of stay for inpatient stays is being predicted. For some individuals there are multiple LOS observations (i.e., more than one IP episode) during the baseline time period which are correlated.

Does it make sense to build, for example, an elastic net predictive model which contains a random effect intercept term for each individual?

$\endgroup$
1
  • $\begingroup$ Subtitles for your first line: "By random effects techniques (ie assumptions about how a parameter is randomly distributed) I'm referring to Laplace priors and Normal priors on regression parameters... and the like" :-) $\endgroup$ Oct 1, 2014 at 16:54

3 Answers 3

20
$\begingroup$

There are a few papers that deal with this question. I would look up in no special order:

  1. Pen.LME: Howard D Bondell, Arun Krishna, and Sujit K Ghosh. Joint variable selection for fixed and random eects in linear mixed-eects models. Biometrics, 66(4):1069-1077, 2010.

  2. GLMMLASSO: Jurg Schelldorfer, Peter Buhlmann, Sara van de Geer. Estimation for high-dimensional linear mixed-eects models using L1- penalization. Scandinavian Journal of Statistics, 38(2):197-214, 2011.

which can be found online.

I happen to be finishing up a paper on applying an elastic net penalty to the mixed model (LMMEN) now and plan to send it for journal review in the coming month.

  1. LMMEN: Sidi, Ritov, Unger. Regularization and Classification of Linear Mixed Models via the Elastic Net Penalty

Over all, if you are modeling data that is either not normal or does not have a identity link I would go with GLMMLASSO, (but beware that it can not handle lots of RE's). Otherwise Pen.LME is good given that you do not have highly correlated data, be it in the fixed or random effects. In the latter case you can mail me and I would be happy to send you code/paper (I will put it on cran in the near future).

I uploaded to CRAN today - lmmen. It solves the linear mixed model problem with an elastic-net type penalty on the fixed and random effects simultaneously.

There is also in the package cv functions for the lmmlasso and glmmLasso packages in it.

$\endgroup$
2
  • 1
    $\begingroup$ Yonicd, the application of elastic net to linear mixed models is exactly what I am looking for. I'm working with genetic data which are correlated, so the grouped selection of an elastic net would be very useful. I've tried to contact you, but find no other way than posting this reply, as it seems impossible to send a message. Anyway, I look forward to reading your paper and will try your code as soon as it's available. $\endgroup$
    – user56773
    Oct 1, 2014 at 16:05
  • $\begingroup$ Seems the lmmen package was deprecated in 2020. As of 2023, there are, fortunately, several packages with penalized lmm's listed on the Mixed Models View $\endgroup$
    – Jthorpe
    Nov 15, 2023 at 18:42
7
$\begingroup$

I always viewed ridge regression as just empirical random effects models not limited to a single categorical variable (and no fancy correlation matrices). You can almost always get the same predictions from cross validating a ridge penalty and fitting/estimating a simple random effect. In your example, you could get fancy and have a separate ridge penalty on the demo/diag features and another one on the patient indicators (using something line the penalty scaling factor in glmnet). Alternatively, you could include a fancy random effect that has time-correlated effects by person. None of these possibilities are right or wrong, they're just useful.

$\endgroup$
2
$\begingroup$

I am currently thinking about a similar question. I think in application, you can do it if it works and you believe using this is reasonable. If it is a usual setting in random effects (that means, you have repeated measurements for each group), then it is just about estimation technique, which is less controversial. If you actually don't have many repeated measurements for most groups, then it might lie on the borderline of usual random effects model and you might want to carefully justify its validity (from a methodology perspective) if you want to proposal it as a general method.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.