I am fairly new to LMM's and I am trying to undestand how the parameter estimation happens; According to this: enter image description here

Beta is obtained with equation 13.28. Beta is supposed to be the parameters for the fixed effects, X stands here for the design matrix for the fixed effects V is the variance-covariance matrix and y is the response value.

However, I don't understand how to obtain matrix V and the book I am using "linear mixed effect models using R by Galecki and Burzykowski is not really clear in it either).

I have tried obtaining the matrix V in R with the following command Cov(Y, X), This returns a vector of 1x7 (which makes sense I have 7 fixed variables; but with this I cannot calculate Beta since V is supposed to be a square matrix.

Any help would be appreciated


First, it helps to consider what dimension the matrix $V_i^{-1}$ needs to have so that the given matrix products are defined. You'll find that it needs to be a square matrix of dimension $n_i \times n_i$, where $n_i$ is the length of the vector $y_i$ (as well as the number of rows of $X_i$). In the book's nomenclature, $y_i$ is the vector of observations in some group $i$. Therefore, it becomes clear that $V_i$ has to be the VCV-matrix of responses in group $i$, $y_i$. (Side note: the estimator in (13.28) is of the form $(X'W^{-1}X')^{-1}X'W^{-1}y$, which would be the Generalized Least Squares (GLS) estimator for $\beta$, if the response $y$ were known to have the VCV-matrix $W$. It is not exactly the GLS estimator, because the VCV-matrix does actually depend on an unknown parameter $\theta$ which has to be estimated.)

This might already help you figure out what matrix V should look like! But the answer is actually also in the book.

Spoiler alert: from (13.25) and (13.26) of the book, you see that $$V_i = \sigma^2(Z_iDZ_i' + R_i),$$ where $Z_i$ is the matrix of regressors for the random effects of group $i$, $D$ is the VCV matrix of random effects, $R_i$ is the correlation matrix of the residuals in group $i$, and $\sigma^2$ is the residual variance. You'll obtain the whole matrix $V$ as a block diagonal matrix with the matrices $V_i$ on the diagonal.

  • $\begingroup$ Thank you for explaining, however some parts that remain vague; you mentioned that σ^2 is the residual variance, In my own data (simulated), I did not add noise hence this is missing and so is R; so than V = ZDZ` ? furthermore the obtained VCV matrix V in my case is square but not a block diagonal..: Here below I will outline what I did to calculate this in R: first I calculated the matrix D simply by calling the cov function on the design matrix for random effects -> cov(Z); note that Z is 45x 15. This in turn gives me a square matrix of 15 x 15. I than obtain V with V= ZDZ' $\endgroup$ – Hedayat Jul 21 '20 at 10:11
  • $\begingroup$ Without further code it is difficult to assess the problems. But I can already tell you one important mistake: D is not equal to cov(Z)! D is a (typically unknown) VCV matrix that specifies the covariance of random effects in the population of groups. It is estimated from the data, and has relatively little to do with Z (although I guess that the estimator for D will in some complicated way depend on Z) $\endgroup$ – Lukas McLengersdorff Jul 21 '20 at 11:50
  • $\begingroup$ Lukas, Thank you for your feedback. I did assume that COV(Z) would result in matrix D. Can you elaborate on how matrix D is obtained? $\endgroup$ – Hedayat Jul 21 '20 at 15:07
  • 1
    $\begingroup$ Estimating matrix D is (IMHO) the central challenge in LMM, and how exactly this works is out of scope for a comment. The book you are already reading should help you further on this. Roughly, all algorithms work by parameterizing D in terms of some parameter, and then maximizing the likelihood (ML) or the restricted likelihood (REML) by means of some optimization algorithm. $\endgroup$ – Lukas McLengersdorff Jul 21 '20 at 17:17
  • $\begingroup$ Ok thanks for all your comments Lukas, really appreciate it! $\endgroup$ – Hedayat Jul 21 '20 at 19:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.