# What is the error covariance structure for linear mixed effects models estimated by lme4?

Recently I read a lot about error covariance structures, especially for longitudinal models. I have found that it is not possible to specify such a structure in lme4. So my question is, what kind of structure does it assume/estimate ? It seems that it's something like $$\delta I$$, where $$I$$ is the identity matrix and $$\delta$$ is the (scalar) error variance which is estimated and shown as part of the random effects output of lme4:::summary.merMod.

Is this correct and, if so, does this structure have any name ?

• This seems like a major disadvantage of R compared to SAS, which offers a large variety of choices. But I wonder if there is a work around. Dec 3, 2023 at 11:36
• @PeterFlom This is a limitation of the lme4 package, not of $\textsf{R}$. Other package, such as nlme, implement R-side effects or covariance structures. Dec 3, 2023 at 13:09
• @statmerkur That makes sense and is good to know. I've retired, but when I was working, I mostly used SAS. These days, I don't have access to it (it's expensive) and my R knowledge is somewhat lacking. Dec 3, 2023 at 13:21
• @PeterFlom I have also read that by fitting random slopes for the time variable, this results in something similar to an AR1 structure. I have probably got that wrong (AR1 that is), I'm not entirely sure what exactly it does do (that was going to be the subject of a further question. I just didn't want to ask too much in one go!) Dec 3, 2023 at 16:53
• I've seen conflicting information but always assumed lme4 did unstructured only. This page shows how you can trick it into fitting certain structured covariances - never tried though & looks complicated. Dec 3, 2023 at 18:49

Yes, this is correct. You can, e.g., read this off from the conditional distribution of the response given the random effects shown in expression $$(2)$$ in this lme4 vignette. Such an error distribution is called "spherical".

lme4 can fit linear mixed models obeying measurement models of the form $$\boldsymbol{\mathcal{Y}} = \boldsymbol X \boldsymbol \beta + \boldsymbol Z \boldsymbol{\mathcal{B}} + \boldsymbol o + \boldsymbol \epsilon,$$ with response vector $$\boldsymbol{\mathcal{Y}} \in \mathbb R^n$$, fixed-effects design matrix $$\boldsymbol X \in \mathbb R^{n \times p}$$, fixed-effects parameter vector $$\boldsymbol \beta \in \mathbb R^p$$, random-effects design matrix $$\boldsymbol Z \in \mathbb R^{n \times q}$$, random-effects vector $$\boldsymbol{\mathcal{B}} \in \mathbb R^q$$, (known) offset vector $$\boldsymbol o \in \mathbb R^n$$, error vector $$\boldsymbol \epsilon \in \mathbb R^n$$;
and the distributional assumption $$\begin{pmatrix} \boldsymbol{\mathcal{B}}\\ \boldsymbol \epsilon \end{pmatrix} \sim \mathop{\mathcal N_{q + n}} \left( \begin{pmatrix} \mathbf 0\\ \mathbf 0 \end{pmatrix}, \begin{pmatrix} \boldsymbol \Sigma_{\boldsymbol \theta} & \mathbf 0\\ \mathbf 0 & \sigma^2 \boldsymbol I_n \end{pmatrix} \right),$$ with positive semi-definite covariance matrix $$\boldsymbol \Sigma_{\boldsymbol \theta} \in \mathbb R^{q \times q}$$ parameterized by the covariance parameter vector $$\boldsymbol \theta \in \mathbb R^m$$.

Such linear mixed models can also be written in the hierarchical form $$\boldsymbol{\mathcal{Y}} \,|\, \boldsymbol{\mathcal{B}} = \boldsymbol b \sim \mathop{\mathcal N_n} \left( \boldsymbol X \boldsymbol \beta + \boldsymbol Z \boldsymbol b + \boldsymbol o, \sigma^2 \boldsymbol I_n \right), \\ \boldsymbol{\mathcal{B}} \sim \mathop{\mathcal N_q} \left( \mathbf 0, \boldsymbol \Sigma_{\boldsymbol \theta} \right),$$ which is the form used in the lme4 vignettes.

From this we can see that the implied distribution of the error vector $$\boldsymbol \epsilon$$ is $$\mathop{\mathcal N_n}\left(\mathbf 0, \sigma^2 \boldsymbol I_n\right)$$, a spherical multivariate normal distribution.

• Does this then mean that we should be checking our data for sphericity before running lmer?
– HarD
Dec 12, 2023 at 14:55
• @HarD I'm not so sure. What would you check ? It's the residuals that have this covariance structure, so how can you obtain the residuals before fitting the model ? Perhaps/hopefully statmerkur will comment Dec 24, 2023 at 17:14
• @statmerkur do you have any comment on HarD's comment ? If we did want to test the residuals, can we, for example, look at a histogram for ALL the residuals lumped together, or do we have to look individually at each subject's residuals ? In the case of lme4 we have only a single estimate for the residual variance, so my guess is that we can look at all the residuals. Is that right ? What about the case with other software that allows heterogeneous variances across subjects - if we lumped all the residuals together and looked at histograms/QQ plots etc, would that be a mistake ? Dec 24, 2023 at 17:20
• @HardD Not necessarily. Sphericity is an assumption that usually concerns the marginal (w.r.t. the random effects) covariance matrix of "subvectors" (corresponding to levels of a grouping factor, e.g., individuals) of the response. And it's possible to specify lmer models whose random effects imply marginal covariance matrices that are more flexible than sphericity. Jan 14 at 13:39
• @Harmony Technically it's not the residuals but the errors that have a spherical covariance matrix. One approach to residual analysis in linear mixed models is to (1) calculated the estimated marginal covariance matrix of the response $\widehat{\boldsymbol{\Sigma}}_{\boldsymbol{\mathcal{Y}}}$, (2) generate the transformed residuals $\widehat{\boldsymbol{\Sigma}}_{\boldsymbol{\mathcal{Y}}}^{-1/2}\left(\boldsymbol{y} - \boldsymbol X \hat{\boldsymbol \beta}\right)$, (3) apply residual diagnostics for the classical linear regression model to these transformed residuals. Jan 14 at 14:25

The older nlme allows to specify various structures for the error terms via its correlation and weights arguments; check the online help file corClasses to see what's available.

The lme4 allows only to model the correlations in the data using random effects. In lme4 you may also use observation-level random effects to allow more flexibility in modeling the error terms. You may find some information in the GLMM FAQ.