I am fairly new to LMM's and I am trying to undestand how the parameter estimation happens; According to this: enter image description here

Beta is obtained with equation 13.28. Beta is supposed to be the parameters for the fixed effects, X stands here for the design matrix for the fixed effects V is the variance-covariance matrix and y is the response value.

However, I don't understand how to obtain matrix V and the book I am using "linear mixed effect models using R by Galecki and Burzykowski is not really clear in it either).

I have tried obtaining the matrix V in R with the following command Cov(Y, X), This returns a vector of 1x7 (which makes sense I have 7 fixed variables; but with this I cannot calculate Beta since V is supposed to be a square matrix.

Any help would be appreciated


1 Answer 1


First, it helps to consider what dimension the matrix $V_i^{-1}$ needs to have so that the given matrix products are defined. You'll find that it needs to be a square matrix of dimension $n_i \times n_i$, where $n_i$ is the length of the vector $y_i$ (as well as the number of rows of $X_i$). In the book's nomenclature, $y_i$ is the vector of observations in some group $i$. Therefore, it becomes clear that $V_i$ has to be the VCV-matrix of responses in group $i$, $y_i$. (Side note: the estimator in (13.28) is of the form $(X'W^{-1}X')^{-1}X'W^{-1}y$, which would be the Generalized Least Squares (GLS) estimator for $\beta$, if the response $y$ were known to have the VCV-matrix $W$. It is not exactly the GLS estimator, because the VCV-matrix does actually depend on an unknown parameter $\theta$ which has to be estimated.)

This might already help you figure out what matrix V should look like! But the answer is actually also in the book.

Spoiler alert: from (13.25) and (13.26) of the book, you see that $$V_i = \sigma^2(Z_iDZ_i' + R_i),$$ where $Z_i$ is the matrix of regressors for the random effects of group $i$, $D$ is the VCV matrix of random effects, $R_i$ is the correlation matrix of the residuals in group $i$, and $\sigma^2$ is the residual variance. You'll obtain the whole matrix $V$ as a block diagonal matrix with the matrices $V_i$ on the diagonal.

  • $\begingroup$ Thank you for explaining, however some parts that remain vague; you mentioned that σ^2 is the residual variance, In my own data (simulated), I did not add noise hence this is missing and so is R; so than V = ZDZ` ? furthermore the obtained VCV matrix V in my case is square but not a block diagonal..: Here below I will outline what I did to calculate this in R: first I calculated the matrix D simply by calling the cov function on the design matrix for random effects -> cov(Z); note that Z is 45x 15. This in turn gives me a square matrix of 15 x 15. I than obtain V with V= ZDZ' $\endgroup$
    – Hedayat
    Jul 21, 2020 at 10:11
  • $\begingroup$ Without further code it is difficult to assess the problems. But I can already tell you one important mistake: D is not equal to cov(Z)! D is a (typically unknown) VCV matrix that specifies the covariance of random effects in the population of groups. It is estimated from the data, and has relatively little to do with Z (although I guess that the estimator for D will in some complicated way depend on Z) $\endgroup$ Jul 21, 2020 at 11:50
  • $\begingroup$ Lukas, Thank you for your feedback. I did assume that COV(Z) would result in matrix D. Can you elaborate on how matrix D is obtained? $\endgroup$
    – Hedayat
    Jul 21, 2020 at 15:07
  • 1
    $\begingroup$ Estimating matrix D is (IMHO) the central challenge in LMM, and how exactly this works is out of scope for a comment. The book you are already reading should help you further on this. Roughly, all algorithms work by parameterizing D in terms of some parameter, and then maximizing the likelihood (ML) or the restricted likelihood (REML) by means of some optimization algorithm. $\endgroup$ Jul 21, 2020 at 17:17
  • $\begingroup$ Ok thanks for all your comments Lukas, really appreciate it! $\endgroup$
    – Hedayat
    Jul 21, 2020 at 19:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.