11
$\begingroup$

I am familiar with notation such as:

\begin{align} y_{ij} &= \beta_0 + \beta_i x_{ij} + u_j + e_{ij}\\ &= \beta_{0j} + \beta_i x_{ij} + e_{ij} \end{align} where $\beta_{0j}=\beta_{0}+u_j$, and

\begin{align} y_{ij} &= \beta_0 + \beta_1 x_{ij} + u_{0j} + u_{1j} x_{ij} + e_{ij} \\ &= \beta_{0j} + \beta_{1j} x_{ij} + e_{ij} \end{align} where $\beta_{0j}=\beta_{0}+u_{0j}$ and $\beta_{1j}=\beta_1+u_{1j}$

for a random intercepts model and a random slope + random intercepts model, respectively.

I have also come across this matrix/vector notation, which I have been told is "mixed model notation for grown ups" (according to my elder brother):

$$ \mathbf{y}=\mathbf{X\beta} + \mathbf{Z b} + \mathbf{e} $$ where $\mathbf{\beta}$ are the fixed effects and $\mathbf{b}$ are the random effects.

If I have understood correctly, the latter notation is a more general notation for the former which are specific versions of the latter.

I should like to see how the former can be derived from the latter.

$\endgroup$
  • 2
    $\begingroup$ Are you asking about an explanation of matrix notation? The reason I ask is that this question doesn't need any mathematical derivation: all your formulas are saying exactly the same things and relating them to one another is just a matter of understanding how matrix notation works. $\endgroup$ – whuber Feb 13 '13 at 21:16
  • $\begingroup$ @whuber I understand matrix notation and matrix algebra, to some extent. But I don't know how to start from the matrix form and arrive at the other forms. Probably I don't understand something about the X and Z matrices, but I was just hoping that someone would spell it out. $\endgroup$ – Joe King Feb 13 '13 at 22:00
  • $\begingroup$ @whuber is there something I can do to improve the question, or are you saying that it's so basic it doesn't deserve an answer ? $\endgroup$ – Joe King Feb 15 '13 at 22:48
  • $\begingroup$ @JoeKing: I think that he's saying that the matrix notation is by definition equivalent to your non-matrix notation. That is, you already have $x_{ij}\beta_i$ (ixj matrix times jx1 matrix yielding ix1 matrix $y_i$) which is $y=X\beta$. (You can roll $\beta_0$ into $\beta$ by including a 1 in $X$.) $\endgroup$ – Wayne Feb 21 '13 at 20:55
  • 2
    $\begingroup$ @Wayne both models have random effects and fixed effects. The first has a random intercept, while the second has a random intercept and a random slope. If I could "figure it out" myself I wouldn't be asking the question here !!!! $\endgroup$ – Joe King Feb 22 '13 at 7:34
13
+50
$\begingroup$

We consider a mixed model with random slopes and random intercepts. Given that we have only one regressor, this model can be written as $$ y_{ij}= \beta_0 + \beta_1 x_{ij} + u_{0j}+u_{1j}x_{ij}+\epsilon_{ij}, $$ where $y_{ij}$ denotes the $i$-th observation of group $j$ of the response, and $x_{ij}$ and $\epsilon_{ij}$ the respective predictor and error term.

This model can be expressed in matrix notation as follows:

$$\mathbf{Y}=\mathbf{X}\beta + \textbf{Zb} + \epsilon,$$ which is equivalent to

$$\mathbf{Y}= \begin{bmatrix} \mathbf{X} & \textbf{Z} \end{bmatrix} \begin{bmatrix} \beta\\ \mathbf{b} \end{bmatrix}+ \epsilon $$

Let us assume that we have $J$ groups, i.e. $j=1,\dots,J$ and let $n_j$ denote the number of observations in the $j$-th group. Partitioned for each group, we can write above formula as

$$\begin{bmatrix} \mathbf{Y_1} \\ \mathbf{Y_2} \\ \vdots \\ \mathbf{Y_J} \end{bmatrix}=\begin{bmatrix} \mathbf{X_1} & \mathbf{Z_1} & 0 & 0 & 0 \\ \mathbf{X_2} & 0 & \mathbf{Z_2} & 0 & 0 \\ \vdots & & & \dots & \\ \mathbf{X_J} & 0 & 0 & 0 & \mathbf{Z_J} \end{bmatrix} \begin{bmatrix} \beta \\ b_1 \\ b_2 \\ \vdots \\ b_J \end{bmatrix}+\begin{bmatrix} \epsilon_1 \\ \epsilon_2 \\ \vdots \\ \epsilon_J \end{bmatrix}$$

where $\mathbf{Y_j}$ is a $n_j \times 1 $ matrix containing all observations of the response for group $j$, $\mathbf{X_j}$ and $\mathbf{Z_j}$ are $n_j \times 2 $ design matrices in this case and $\epsilon_j$ is again a $n_j \times 1 $ matrix.

Writing them out, we have:

$\mathbf{Y_j} = \begin{bmatrix} y_{1j}\\ y_{2j}\\ \vdots \\ y_{n_jj} \end{bmatrix}, \mathbf{X_j}=\mathbf{Z_j}=\begin{bmatrix} 1 & x_{1j} \\ 1 & x_{2j} \\ \vdots & \vdots \\ 1 & x_{n_jj} \end{bmatrix}$ and $\epsilon_j = \begin{bmatrix} \epsilon_{1j}\\ \epsilon_{2j}\\ \vdots \\ \epsilon_{n_jj} \end{bmatrix}.$

The regression coefficient vectors then are

$\beta = \begin{pmatrix} \beta_0 \\ \beta_1 \end{pmatrix}$, $b_j=\begin{pmatrix} u_{0j}\\ u_{1j} \end{pmatrix}$

To see that the two model formulations are indeed equivalent, let us look at any of the groups (let's say the $j$-th one).

$$ \mathbf{Y_j} = \mathbf{X_j} \beta + \mathbf{Z_j}b_j + \epsilon_j$$

Applying above definitions, one can show that the $i$-th row of the resulting vector is just $$ y_{ij}= \beta_0 + \beta_1 x_{ij} + u_{0j}+u_{1j}x_{ij}+\epsilon_{ij}, $$ where $i$ ranges from $1$ to $n_j$.

$\endgroup$
  • 1
    $\begingroup$ +1, I would just point out that there are large computational advantages from implementing using $Z_j$ rather than the full $Z$ matrix. The $Z_j$ are basically a sparse matrix storage version of $Z$ $\endgroup$ – probabilityislogic Feb 25 '13 at 8:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.