How to choose nlme or lme4 R library for mixed effects models?

Question

I have fit a few mixed effects models (particularly longitudinal models) using lme4 in R but would like to really master the models and the code that goes with them.

However, before diving in with both feet (and buying some books) I want to be sure that I am learning the right library. I have used lme4 up to now because I just found it easier than nlme, but if nlme is better for my purposes then I feel I should use that.

I'm sure neither is "better" in a simplistic way, but I would value some opinions or thoughts. My main criteria are:

1. Easy to use (I'm a psychologist by training, and not particularly versed in statistics or coding, but I'm learning)
2. Good features for fitting longitudinal data (if there is a difference here- but this is what I mainly use them for)
3. Good (easy to interpret) graphical summaries, again not sure if there is a difference here but I often produce graphs for people even less technical than I, so nice clear plots are always good (I'm very fond of the xyplot function in lattice() for this reason).

Answer 1

Both packages use Lattice as the backend, but nlme has some nice features like groupedData() and lmList() that are lacking in lme4 (IMO). From a practical perspective, the two most important criteria seem, however, that

1. lme4 extends nlme with other link functions: in nlme, you cannot fit outcomes whose distribution is not gaussian, lme4 can be used to fit mixed-effects logistic regression, for example.
2. in nlme, it is possible to specify the variance-covariance matrix for the random effects (e.g. an AR(1)); it is not possible in lme4.

Now, lme4 can easily handle very huge number of random effects (hence, number of individuals in a given study) thanks to its C part and the use of sparse matrices. The nlme package has somewhat been superseded by lme4 so I won't expect people spending much time developing add-ons on top of nlme. Personally, when I have a continuous response in my model, I tend to use both packages, but I'm now versed to the lme4 way for fitting GLMM.

Rather than buying a book, take a look first at the Doug Bates' draft book on R-forge: lme4: Mixed-effects Modeling with R.

Answer 2

As chl pointed out, the main difference is what kind of variance-covariance structure you can specify for the random effects. In lme4 you can specify either:

• diagonal covariance structures (i.e., enforce mutually uncorrelated random effects via syntax like ~ (1 | group)+ (0 + x1 | group) + (0 + x2 | group))
• or unstructured covariance matrices (i.e. all correlations are estimated, ~ (1 + x1 + x2 | group))
• or partially diagonal, partially unstructured covariance (y ~ (1 + x1 | group) + (0 + x2 | group), where you would estimate a correlation between the random intercept and random slope for x1, but no correlations between the random slope for x2 and the random intercept and between the random slope for x2 and the random slope for x1).

nlme offers a much broader class of covariance structures for the random effects. My experience is that the flexibility of lme4 is sufficient for most applications, however.

I'd also add a third difference in capabilities that may be more relevant for many longitudinal data situations: nlme let's you specify variance-covariance structures for the residuals (i.e. spatial or temporal autocorrelation or heteroskedasticity or covariate-dependent variability) in the weights argument (c.f. ?varFunc), while lme4 only allows fixed prior weights for the observations.

A fourth difference is that it can be difficult to get nlme to fit (partially) crossed random effects, while that's a non-issue in lme4.

You'll probably be fine if you stick with lme4.

Answer 3

Others have summarized the differences very well. My impression is that lme4 is more suited for clustered data sets especially when you need to use crossed random effects. For repeated measures designs (including many longitudinal designs) however, nlme is the tool since only nlme supports specifying a correlation structure for the residuals. You do it using the correlations or cor argument with a corStruct object. It also allows you to model heteroscedasticity using a varFunc object.

Answer 4

There are actually a number of packages in R for fitting mixed effects models beyond lme4 and nlme. There's a nice wiki run by the R special interest group for mixed models, which has a very nice FAQ and a page comparing the different packages.

As for my opinions on actually using lme4 and nlme: I found lme4 to be generally easier to use due to its rather direct extension of the basic R formula syntax. (If you need to work with generalized additive models, then the gamm4 package extends this syntax one further step and so you have a nice smooth learning curve.) As others have mentioned, lme4 can handle generalized models (other link functions and error distributions), while nlme's focus on the Gaussian link function allows it do so some things that are very hard in the general case (specifying covariance structure and certain things dependent on degrees of freedom calculation, like p-values, the latter of which I encourage you to move away from!).