lme and lmer comparison

Question

I was wondering if anyone could enlighten me on the current differences between these two functions. I found the following question: How to choose nlme or lme4 R library for mixed effects models?, but that dates from a couple of years ago. That's a lifetime in software circles.

My specific questions are:

• Are there (still) any correlation structures in lme that lmer doesn't handle?
• Is it possible/recommended to use lmer for panel data?

Apologies if these are somewhat basic.

A bit more detail: panel data is where we have multiple measurements on the same individuals, at different points in time. I generally work in a business context, where you might have data for repeat/long-term customers over a number of years. We want to allow for variation over time, but clearly fitting a dummy variable for each month or year is inefficient. However, I'm unclear whether lmer is the appropriate tool for this sort of data, or whether I need the autocorrelation structures that lme has.

Answer 1

UPDATE JUNE 2016:

Please see Ben's blog entry describing his current thoughts on accomplishing this in lme4: Braindump 01 June 2016

If you prefer Bayesian methods, the brms package's brm supports some correlation structures: CRAN brms page. (Note especially: "As of brms version 0.6.0, the AR structure refers to autoregressive effects of residuals to match the naming and implementation in other packages such as nlme. Previously, the AR term in brms referred to autoregressive effects of the response. The latter are now named ARR effects and can be modeled using argument r in the cor_arma and cor_arr functions.")

---

ORIGINAL ANSWER JULY 2013:

(Converted from a comment.)

I would say lmer would be pretty good with a random effect of year and a random effect of customer (let's say you only have one measurement per customer per year);

lmer(y~1 + (1|year) + (1|customer), ...)

would fit the (intercept-only) model

$$ Y_{ij} \sim \text{Normal}(a + \epsilon_{\text{year},i} + \epsilon_{\text{customer},j}, \sigma^2_0) $$ where $\epsilon_{\text{year}}$ and $\epsilon_{\text{customer}}$ are zero-mean Normal variates with their own specific variances.

This is a pretty boring model, you might want to add an overall (fixed-effect) trend of time and also consider a random time-by-customer interaction (i.e. random slopes). I think

lmer(y~year + (1|year) + (year|customer), ...)

should fit the model $$ Y_{ij} \sim \text{Normal}((a + \epsilon_{\text{customer},j}) + (b + \epsilon_{\text{year} \times \text{customer},j}) \cdot \text{year} + \epsilon_{\text{year},i}, \sigma^2_0) $$

(using year in this way is an exception to the usual rule of not including an input variable as both a fitted and a random effect in the same model; provided it's a numeric variable, year gets treated as continuous in the fixed effect and the year:customer (random) interaction and as categorical in the random effect ...)

Of course you might want to add year-level, customer-level, and observation-level covariates which would soak up some of the relevant variance (e.g. add average consumer price index to explain why years were bad or good ...)

Ideally you would also want to allow for temporal autocorrelation within each customer's time series, which is at the moment not possible with lmer, but you could check the temporal autocorrelation function to see if that was important ...

Caveat: I don't know that much about standard approaches for handling panel data; this is based just on my knowledge of mixed models. Commenters (or editors) should feel free to chime in if this seems to violate standard/best practices in econometrics.

Answer 2

To answer your questions directly, and NB this is years after the original post!

• Yep there are still correlation structures that nlme handles which lme4 will not handle. However, for as long as nlme allows the user to define general corstrs and lme4 does not, this will be the case. This has surprisingly little practical impact. The "big three" correlation structures of: Independent, Exchangeable, and AR-1 correlation structures are easy handled by both packages.

• It's certainly possible. You can fit panel data with the lm function too! My recommendation about which to use depends on the problem. lme4 is a much smaller tool kit, and the formula representation is a neat, concise way of depicting some very common mixed effects models. nlme is the very large tool box, including a TIG welder to make any tools you need.

You say you want to allow for "variation over time". Essentially, an exchangeable correlation structure achieves this, allowing for a random intercept in each cluster, so that the intracluster variance is the sum of cluster level variation as well as (what you call) variation over time. And this by no means deters you from using fixed effects to obtain more precise predictions over time.