In parameter estimation for linear mixed model for unknown variance, I met some statements saying that "we assume G (as variance) is only known up to its variance parameter v", then, G is represented as G(v). I met this statement in the book Generalized, Linear, and Mixed Models. Examples of these statements can be found in page 12 in this one.

I wonder what is the relationship between G and v? To estimate G, we need to maximize a likelihood function of G(v), i.e.

l(v) = XG(v)Y

(of course the actual likelihood is much more complicated than this simple one)

So, I wonder how we could write G(v) as a function of v explicitly, so that we could maximize it?

A similar question has been asked here, but I am looking for a more concrete solution, which I assume will be helpful for the ones who are interesting in implementing their own linear mixed model.


1 Answer 1


Let $\mathbf b$ denote random effects, as the common notation in literature. Equation 1.1 in the book defines $$\mathrm{var}(\mathbf b)=\Sigma_\theta.$$ Then Equation 1.2 does some Cholesky-like decomposition $$\Sigma_\theta=\sigma^2 \Lambda_\theta \Lambda_\theta^T.$$

The author call $\theta$ as "variance-component parameter". In my opinion, $\theta$ denotes all parameters in the variance-covariance matrix of random effects, $\Sigma_\theta$, including $\sigma^2$ in Equation 1.2.

@AlaskaRon did answer the question you linked, with a simplest example, i.e., random intercept model. If there are $q$ random effects, $$\Sigma_\theta=\sigma^2 \mathbf I_q.$$ Thus $\theta=\sigma^2$ or there is no parameter to be estimated in $\Lambda_\theta$ at all.

Also as @deep-north commented, "$\theta$ just means what kind of variance-component you will assume, such as AR(1) or UN, etc." though auto-correlated random effects do not make much sense to me. The most general case is the unstructured (UN) $\Sigma_\theta$. Then $\Lambda_\theta$ probably would be triangular, and $\theta$ would be all nonzero elements therein.

For me, it seems more natural to think this question within each cluster. Let $\Sigma_\theta$ be the variance-covariance matrix for random effects $\mathbf b_i$ for cluster $i$. For random intercept model, $\Sigma_\theta$ is just a scalar, i.e., the variance of random intercept. For multiple random effects model, we can re-parameterize $\Sigma_\theta$ following Equation 1.2, then $\theta$ in $\Lambda_\theta$ would be all elements on and below the diagonal of $\Lambda_\theta$.

In summary, $\theta$ depends on your variance-covariance structure for random effects and your parameterization.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.