# Linear Mixed Model equation (as of lme4 package)

I am trying to derive the equations of a linear mixed model as specified in the documentation of the lme4 package: "Fitting Linear Mixed-Effects Models using lme4" jstatsoft.org/article/view/v067i01

In the paper, the model is described by the conditional distribution of a vector-valued variable, $$\mathcal{Y}$$, given the vector of random effects $$\mathcal{B} = \boldsymbol{b}$$:

$$\begin{equation*} (\mathcal{Y} | \mathcal{B} = {\bf b}) \sim N({\boldsymbol \mu}_{\mathcal{Y} | \mathcal{B} = {\bf b}}, \sigma^2{\bf W}^{-1})\ , \end{equation*}$$

$$\begin{equation*} {\boldsymbol \mu}_{\mathcal{Y} | \mathcal{B} = {\bf b}} = {\bf X \beta} + {\bf Z b} + {\bf o}\ , \end{equation*}$$

and, at some point, the authors say that is preferable to work with a spherical random effects variable $$\mathcal{U}$$, such that

$$\begin{equation*} \mathcal{U} \sim N(\boldsymbol{0}, \sigma^2\boldsymbol{I}_q)\ , \end{equation*}$$

instead of dealing with the vector of random effects $$\mathcal{B}$$. Then the model would be described by:

$$\begin{equation*} (\mathcal{Y} | \mathcal{U} = {\bf u}) \sim N({\boldsymbol \mu}_{\mathcal{Y} | \mathcal{U} = {\bf u}}, \sigma^2{\bf W}^{-1})\ , \end{equation*}$$

$$\begin{equation*} {\boldsymbol \mu}_{\mathcal{Y} | \mathcal{U} = {\bf u}} = {\bf X \beta} + {\bf Z}\boldsymbol{\Lambda_{\theta}u} + {\bf o}\ , \end{equation*}$$

I can understand that it's better to use something we know (a spherical variable in this case). But, in the text, it's mentioned that we should define $$\mathcal{U}$$ as

$$\begin{equation} \mathcal{B} = \boldsymbol{\Lambda}_{\boldsymbol{\theta}}\mathcal{U}\ , \end{equation}$$

since the above equation "allows for singular $$\Lambda_{\theta}$$".

My questions are the following:

• What is the meaning of "allows for singular $$\Lambda_{\theta}$$"?

• Why does $$\Lambda_{\theta}$$ needs to be singular?

My questions are the following:

• What is the meaning of "allows for singular $$\Lambda_{\theta}$$"?

This has to do with computational stability. The objective function implements a penalized least squares algorithm, which is optimized over the covariance parameters $$\theta$$. The reformulation improves stability, because the solution to the PLS problem is well-defined even when $$\Lambda_{\theta}$$ is singular.

• Why does $$\Lambda_{\theta}$$ needs to be singular?

It does not need to be singular. The point is that if it is singular, the optimization can still proceed.

• +1. I would also note that one of the main (numerical) linear algebra tricks behind lmer was/is that use of the Cholesky decomposition (eq. 18 in the paper). They CHOLMOD that bad-boy to oblivion and though $LDL^T$ they are able to get past Positive Semi-Definite matrices fast and efficiently. – usεr11852 Jul 10 '19 at 10:38
• @usεr11852 thanks. I think you meant "get past NON-positive semi-definite matrices" though ? – Robert Long Jul 10 '19 at 19:08
• Won't that be indefinite then? (Playing with words here) I think they are still working with PSD matrices only. $\Lambda_\theta$ is a (relative) covariance matrix after all. The whole equation can go indefinite (it is block-symmetric in any case) but those would take some very degenerate $\theta$ that have no physical interpretation (e.g. within subject variance being greater than total sample variance and similar crazy stuff). (I had coded a primitive MVLMER years ago but my memory is not 100% on this.) – usεr11852 Jul 10 '19 at 22:53