3
$\begingroup$

I am trying to derive the equations of a linear mixed model as specified in the documentation of the lme4 package: "Fitting Linear Mixed-Effects Models using lme4" jstatsoft.org/article/view/v067i01

In the paper, the model is described by the conditional distribution of a vector-valued variable, $\mathcal{Y}$, given the vector of random effects $\mathcal{B} = \boldsymbol{b}$:

\begin{equation*} (\mathcal{Y} | \mathcal{B} = {\bf b}) \sim N({\boldsymbol \mu}_{\mathcal{Y} | \mathcal{B} = {\bf b}}, \sigma^2{\bf W}^{-1})\ , \end{equation*}

\begin{equation*} {\boldsymbol \mu}_{\mathcal{Y} | \mathcal{B} = {\bf b}} = {\bf X \beta} + {\bf Z b} + {\bf o}\ , \end{equation*}

and, at some point, the authors say that is preferable to work with a spherical random effects variable $\mathcal{U}$, such that

\begin{equation*} \mathcal{U} \sim N(\boldsymbol{0}, \sigma^2\boldsymbol{I}_q)\ , \end{equation*}

instead of dealing with the vector of random effects $\mathcal{B}$. Then the model would be described by:

\begin{equation*} (\mathcal{Y} | \mathcal{U} = {\bf u}) \sim N({\boldsymbol \mu}_{\mathcal{Y} | \mathcal{U} = {\bf u}}, \sigma^2{\bf W}^{-1})\ , \end{equation*}

\begin{equation*} {\boldsymbol \mu}_{\mathcal{Y} | \mathcal{U} = {\bf u}} = {\bf X \beta} + {\bf Z}\boldsymbol{\Lambda_{\theta}u} + {\bf o}\ , \end{equation*}

I can understand that it's better to use something we know (a spherical variable in this case). But, in the text, it's mentioned that we should define $\mathcal{U}$ as

\begin{equation} \mathcal{B} = \boldsymbol{\Lambda}_{\boldsymbol{\theta}}\mathcal{U}\ , \end{equation}

since the above equation "allows for singular $\Lambda_{\theta}$".

My questions are the following:

  • What is the meaning of "allows for singular $\Lambda_{\theta}$"?

  • Why does $\Lambda_{\theta}$ needs to be singular?

$\endgroup$
2
$\begingroup$

My questions are the following:

  • What is the meaning of "allows for singular $\Lambda_{\theta}$"?

This has to do with computational stability. The objective function implements a penalized least squares algorithm, which is optimized over the covariance parameters $\theta$. The reformulation improves stability, because the solution to the PLS problem is well-defined even when $\Lambda_{\theta}$ is singular.

  • Why does $\Lambda_{\theta}$ needs to be singular?

It does not need to be singular. The point is that if it is singular, the optimization can still proceed.

$\endgroup$
  • 1
    $\begingroup$ +1. I would also note that one of the main (numerical) linear algebra tricks behind lmer was/is that use of the Cholesky decomposition (eq. 18 in the paper). They CHOLMOD that bad-boy to oblivion and though $LDL^T$ they are able to get past Positive Semi-Definite matrices fast and efficiently. $\endgroup$ – usεr11852 says Reinstate Monic Jul 10 at 10:38
  • $\begingroup$ @usεr11852 thanks. I think you meant "get past NON-positive semi-definite matrices" though ? $\endgroup$ – Robert Long Jul 10 at 19:08
  • $\begingroup$ Won't that be indefinite then? (Playing with words here) I think they are still working with PSD matrices only. $\Lambda_\theta$ is a (relative) covariance matrix after all. The whole equation can go indefinite (it is block-symmetric in any case) but those would take some very degenerate $\theta$ that have no physical interpretation (e.g. within subject variance being greater than total sample variance and similar crazy stuff). (I had coded a primitive MVLMER years ago but my memory is not 100% on this.) $\endgroup$ – usεr11852 says Reinstate Monic Jul 10 at 22:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.