In parameter estimation for linear mixed model for unknown variance, I met some statements saying that "we assume G (as variance) is only known up to its variance parameter v", then, G is represented as G(v). I met this statement in the book Generalized, Linear, and Mixed Models. Examples of these statements can be found in page 12 in this one.

I wonder what is the relationship between G and v? To estimate G, we need to maximize a likelihood function of G(v), i.e.

l(v) = XG(v)Y

(of course the actual likelihood is much more complicated than this simple one)

So, I wonder how we could write G(v) as a function of v explicitly, so that we could maximize it?

A similar question has been asked here, but I am looking for a more concrete solution, which I assume will be helpful for the ones who are interesting in implementing their own linear mixed model.


Let $\mathbf b$ denote random effects, as the common notation in literature. Equation 1.1 in the book defines $$\mathrm{var}(\mathbf b)=\Sigma_\theta.$$ Then Equation 1.2 does some Cholesky-like decomposition $$\Sigma_\theta=\sigma^2 \Lambda_\theta \Lambda_\theta^T.$$

The author call $\theta$ as "variance-component parameter". In my opinion, $\theta$ denotes all parameters in the variance-covariance matrix of random effects, $\Sigma_\theta$, including $\sigma^2$ in Equation 1.2.

@AlaskaRon did answer the question you linked, with a simplest example, i.e., random intercept model. If there are $q$ random effects, $$\Sigma_\theta=\sigma^2 \mathbf I_q.$$ Thus $\theta=\sigma^2$ or there is no parameter to be estimated in $\Lambda_\theta$ at all.

Also as @deep-north commented, "$\theta$ just means what kind of variance-component you will assume, such as AR(1) or UN, etc." though auto-correlated random effects do not make much sense to me. The most general case is the unstructured (UN) $\Sigma_\theta$. Then $\Lambda_\theta$ probably would be triangular, and $\theta$ would be all nonzero elements therein.

For me, it seems more natural to think this question within each cluster. Let $\Sigma_\theta$ be the variance-covariance matrix for random effects $\mathbf b_i$ for cluster $i$. For random intercept model, $\Sigma_\theta$ is just a scalar, i.e., the variance of random intercept. For multiple random effects model, we can re-parameterize $\Sigma_\theta$ following Equation 1.2, then $\theta$ in $\Lambda_\theta$ would be all elements on and below the diagonal of $\Lambda_\theta$.

In summary, $\theta$ depends on your variance-covariance structure for random effects and your parameterization.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.