# How to choose nlme or lme4 R library for mixed effects models?

I have fit a few mixed effects models (particularly longitudinal models) using lme4 in R but would like to really master the models and the code that goes with them.

However, before diving in with both feet (and buying some books) I want to be sure that I am learning the right library. I have used lme4 up to now because I just found it easier than nlme, but if nlme is better for my purposes then I feel I should use that.

I'm sure neither is "better" in a simplistic way, but I would value some opinions or thoughts. My main criteria are:

1. easy to use (I'm a psychologist by training, and not particularly versed in statistics or coding, but I'm learning)
2. good features for fitting longitudinal data (if there is a difference here- but this is what I mainly use them for)
3. good (easy to interpret) graphical summaries, again not sure if there is a difference here but I often produce graphs for people even less technical than I, so nice clear plots are always good (I'm very fond of the xyplot function in lattice() for this reason).

As usual, hope this question isn't too vague, and thanks in advance for any wisdom!

-

Both packages use Lattice as the backend, but nlme has some nice features like groupedData() and lmList() that are lacking in lme4 (IMO). From a practical perspective, the two most important criteria seem, however, that

1. lme4 extends nlme with other link functions: in nlme, you cannot fit outcomes whose distribution is not gaussian, lme4 can be used to fit mixed-effects logitic regression, for example.
2. in nlme, it is possible to specify the variance-covariance matrix for the random effects (e.g. an AR(1)); it is not possible in lme4.

Now, lme4 can easily handle very huge number of random effects (hence, No. individuals in a given study) thanks to its C part and the use of sparse matrices. The nlme package has somewhat been superseded by lme4 so I won't expect people spending much time developing add-ons on top of nlme. Personally, when I have a continuous response in my model, I tend to use both packages, but I'm now versed to the lme4 way for fitting GLMM.

Rather than buying a book, take a look first at the Doug Bates' draft book on R-forge: lme4: Mixed-effects Modeling with R.

-
@ 2) more precisely, in lme4 you can either specify a diagonal covariance structure (i.e., independent random effects) or unstructured covariance matrices (i.e. all correlations have to be estimated) or partially diagonal, partially unstructured covariance matrices for the random effects. I'd also add a third difference in capabilities that may be more relevant for many longitudinal data situations: nlme let's you specify variance-covariance structures for the residuals (i.e. spatial or temporal autocorrelation or heteroskedasticity), lme4 doesn't. –  fabians Dec 10 '10 at 11:55
@fabians (+1) Ah, thanks! Didn't realize lme4 allows to choose different VC structures. It would be better that you add it in your own response, together with other ideas you may have. I will upvote. BTW, I also realized that lmList() is available in lme4 too. I seem to remember some discussion about that on R-sig-ME. –  chl Dec 10 '10 at 12:09
what's the etiquette here - should I delete my comment now? –  fabians Dec 10 '10 at 13:19
@fabians I was thinking that your comments would in themselves constitute a response, in addition to other ideas that you may have (because it seems you have some good experience with GLMM). Feel free to leave them as is or to develop them in a new post (that I will vote up because I appreciated your remarks). Sorry if I was unclear or if my preceding comment looks offending or whatever. –  chl Dec 10 '10 at 14:04
@fabians Ah... I just realized that you answered in the meantime. Forget about that. –  chl Dec 10 '10 at 14:12

As chl pointed out, the main difference is what kind of variance-covariance structure you can specify for the random effects. In lme4 you can specify either:

• diagonal covariance structures (i.e., enforce mutually uncorrelated random effects via syntax like ~ (1 | group)+ (0 + x1 | group) + (0 + x2 | group))
• or unstructured covariance matrices (i.e. all correlations are estimated, ~ (1 + x1 + x2 | group))
• or partially diagonal, partially unstructured covariance (y ~ (1 + x1 | group) + (0 + x2 | group), where you would estimate a correlation between the random intercept and random slope for x1, but no correlations between the random slope for x2 and the random intercept and between the random slope for x2 and the random slope for x1).

nlme offers a much broader class of covariance structures for the random effects. My experience is that the flexibility of lme4 is sufficient for most applications, however.

I'd also add a third difference in capabilities that may be more relevant for many longitudinal data situations: nlme let's you specify variance-covariance structures for the residuals (i.e. spatial or temporal autocorrelation or heteroskedasticity or covariate-dependent variability) in the weights argument (c.f. ?varFunc), while lme4 only allows fixed prior weights for the observations.

A fourth difference is that it can be difficult to get nlme to fit (partially) crossed random effects, while that's a non-issue in lme4.

You'll probably be fine if you stick with lme4.

-
With the possible exception (as you pointed out) of being able to incorporate temporal autocorrelation in nlme but not lme4. If the data set is big enough, and if the data have this sort of structure, that could be a big advantage of nlme. –  Ben Bolker Dec 10 '10 at 21:10
Others have summarized the differences very well. My impression is that lme4 is more suited for clustered data sets especially when you need to use crossed random effects. For repeated measures designs (including many longitudinal designs) however, nlme is the tool since only nlme supports specifying a correlation structure for the residuals. You do it using the correlations or cor argument with a corStruct object. It also allows you to model heteroscedasticity using a varFunc object.