Fitting special variance structure in mixed model

Question

I'm interested in fitting a linear mixed model with this special variance structure on the random effects $\mathbf{u}$:

$\begin{eqnarray*} \mathbb{V}\left(\mathbf{u}\right) & = & \mathbf{A}\mathbf{G}\mathbf{A}^{\prime} \end{eqnarray*}$

This variance structure is very similar to Cholesky and Antedependence structures except few differences:

1. $\mathbf{A}$ is not a unit lower triangle or unit upper triangle matrix
2. $\mathbf{G}$ is not necessarily a diagonal matrix

I can specify initial values of $\mathbf{A}$. I'd highly appreciate if someone give me some hints to fit this variance covariance structure in R. Thanks in advance for your help and time.

Answer 1

With no restrictions on $A$, your equation doesn't place any restrictions whatsoever on the form of the variance-covariance matrix, even if you do specify that $G$ is diagonal. A variance-covariance matrix is a real symmetric matrix, and any such matrix can be diagonalized by an orthogonal matrix.

So you can just fit your model with an unstructured variance-covariance matrix.