I see models for "mixed effects" (i.e., models with fixed as well as random factors) specified in the literature in two ways, and I'd like to understand the conditions under which they are equivalent...

Model Formulation 1:

$$ Y = \ X \beta + \ Zb +\epsilon $$ $$ \epsilon \thicksim N(0,\sigma^2 I) $$ $$ b \thicksim N(0,\Sigma)$$ $$ Cov(\epsilon,b)=0 $$ Here the random effects are specified explicitly. The distribution for $ Y $, which can be written $Y \thicksim N(X \beta+\ Z b, \sigma^2 I)$, is sometimes called "conditional on $b$."

Model Formulation 2:

$$ Y \thicksim N(X \beta, \Gamma) $$ $$ \Gamma = Z \Sigma Z' + \sigma^2 I $$

Here, the random effects are specified implicitly via $Z$ and the elements of covariance matrix $ \Gamma $, the expression for which can be derived from the assumptions in Model 1. This formulation is like that used for "Generalized Least Squares."

These two different formulations are sometimes used interchangeably. E.g., Rencher and Schaalje, 2008. On page 480, the mixed model specification is like Model 1, whereas on page 486, expression (17.3), it is like Model 2. In the latter case, it is being used for the exposition of residual maximum likelihood (REML).

My concern is, in using Model 2 instead of Model 1, is one giving up some degrees of freedom in the calculations? Or are these two formulations completely equivalent, i.e., both leading to the same results for $\Sigma$ and $\beta $ as a function of the observed data for $Y$, $X$, and $Z$?

  • $\begingroup$ I think your understanding is correct, you are simply missing the intermediate conceptual step; please see my answer below. $\endgroup$
    – usεr11852
    Jan 23 '16 at 0:37

While I think your understanding is generally correct you appear to miss a small conceptual step. So let's say you have a fixed effects model with normal errors:

\begin{align} y = X\beta + \epsilon \leftrightarrow y \sim N(X\beta, \sigma^2 I) \end{align}

where $X$ is your model matrix and $\beta$ your parameter vector. You generalise this model by assuming some random effects $\gamma$ associated with a model matrix $Z$. So you get something like what you labeled as Model formulation 1:

\begin{align} y = X\beta + Z\gamma + \epsilon \leftrightarrow y|\gamma \sim N(X\beta+Z\gamma, \sigma^2 I) \end{align}

Now assuming that $\gamma \sim N(0, \Sigma_\gamma)$ that means that Var($y$) = Var($Z\gamma$) + Var($\epsilon$) = $Z \Sigma_\gamma Z^T + \sigma^2 I$. As such the unconditional distribution of $y$ is what you labeled as Model formulation 2:

\begin{align} y \sim N(X\beta, Z \Sigma_\gamma Z^T +\sigma^2 I) \end{align}

So yes, these two model formulations are equivalent.

  • $\begingroup$ But does the choice of formulation constrain one's calculation methodologies and restrict availability of some info such as conditional modes of random factors? I.e., see Bates' vignette for the R lme4 package, PLSvsGLS.pdf regarding computation. Also, though you've shown Formulation 1 implies Formulation 2, does the reverse hold? Many thanks for your interest and efforts! $\endgroup$
    – ClarPaul
    Jan 23 '16 at 1:01
  • $\begingroup$ The choice of formulation definitely affects ones methodology because it changes the problem's expression; lmer is great because it really transforms this LMER task to a sparse Cholesky decomposition. I think 'PLSvsGLS.pdf' is too sketchy, it always confused as a student; try Bates and DebRoyb 2004 'Linear mixed models and penalized least squares' first. If anything though the 'PLSvsGLS' (tries to) show(s) that both formulations can be expressed through a Cholesky factor (and thus yield the same solution). Yes the reverse holds you just "decompose" the variance instead of "concatenating" it. $\endgroup$
    – usεr11852
    Jan 23 '16 at 3:33
  • $\begingroup$ @user11852, what is meant by 'decompose' vs. 'concatenate'? $\endgroup$
    – ClarPaul
    Jan 23 '16 at 3:41
  • $\begingroup$ Sorry for being cryptic, my bad; I was running out of space (and time). :) By 'decompose' I meant that you recognise that the variance Var($\epsilon$) has an association with a model matrix $Z$ so you break it into Var($Z\gamma$) + Var($\epsilon$). By concatenation I mean the opposite; you recognise that you variance components are are Gaussian and you add them up to get another 'concatenated' one. The book Variance Components - Chapt. 8 by Searle et al. is great in explaining these issues. $\endgroup$
    – usεr11852
    Jan 23 '16 at 7:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.