# Reconciling notations for mixed models

I am familiar with notation such as:

\begin{align} y_{ij} &= \beta_0 + \beta_i x_{ij} + u_j + e_{ij}\\ &= \beta_{0j} + \beta_i x_{ij} + e_{ij} \end{align} where $\beta_{0j}=\beta_{0}+u_j$, and

\begin{align} y_{ij} &= \beta_0 + \beta_1 x_{ij} + u_{0j} + u_{1j} x_{ij} + e_{ij} \\ &= \beta_{0j} + \beta_{1j} x_{ij} + e_{ij} \end{align} where $\beta_{0j}=\beta_{0}+u_{0j}$ and $\beta_{1j}=\beta_1+u_{1j}$

for a random intercepts model and a random slope + random intercepts model, respectively.

I have also come across this matrix/vector notation, which I have been told is "mixed model notation for grown ups" (according to my elder brother):

$$\mathbf{y}=\mathbf{X\beta} + \mathbf{Z b} + \mathbf{e}$$ where $\mathbf{\beta}$ are the fixed effects and $\mathbf{b}$ are the random effects.

If I have understood correctly, the latter notation is a more general notation for the former which are specific versions of the latter.

I should like to see how the former can be derived from the latter.

• Are you asking about an explanation of matrix notation? The reason I ask is that this question doesn't need any mathematical derivation: all your formulas are saying exactly the same things and relating them to one another is just a matter of understanding how matrix notation works.
– whuber
Feb 13, 2013 at 21:16
• @whuber I understand matrix notation and matrix algebra, to some extent. But I don't know how to start from the matrix form and arrive at the other forms. Probably I don't understand something about the X and Z matrices, but I was just hoping that someone would spell it out. Feb 13, 2013 at 22:00
• @whuber is there something I can do to improve the question, or are you saying that it's so basic it doesn't deserve an answer ? Feb 15, 2013 at 22:48
• @JoeKing: I think that he's saying that the matrix notation is by definition equivalent to your non-matrix notation. That is, you already have $x_{ij}\beta_i$ (ixj matrix times jx1 matrix yielding ix1 matrix $y_i$) which is $y=X\beta$. (You can roll $\beta_0$ into $\beta$ by including a 1 in $X$.) Feb 21, 2013 at 20:55
• @Wayne both models have random effects and fixed effects. The first has a random intercept, while the second has a random intercept and a random slope. If I could "figure it out" myself I wouldn't be asking the question here !!!! Feb 22, 2013 at 7:34

We consider a mixed model with random slopes and random intercepts. Given that we have only one regressor, this model can be written as $$y_{ij}= \beta_0 + \beta_1 x_{ij} + u_{0j}+u_{1j}x_{ij}+\epsilon_{ij},$$ where $y_{ij}$ denotes the $i$-th observation of group $j$ of the response, and $x_{ij}$ and $\epsilon_{ij}$ the respective predictor and error term.

This model can be expressed in matrix notation as follows:

$$\mathbf{Y}=\mathbf{X}\beta + \textbf{Zb} + \epsilon,$$ which is equivalent to

$$\mathbf{Y}= \begin{bmatrix} \mathbf{X} & \textbf{Z} \end{bmatrix} \begin{bmatrix} \beta\\ \mathbf{b} \end{bmatrix}+ \epsilon$$

Let us assume that we have $J$ groups, i.e. $j=1,\dots,J$ and let $n_j$ denote the number of observations in the $j$-th group. Partitioned for each group, we can write above formula as

$$\begin{bmatrix} \mathbf{Y_1} \\ \mathbf{Y_2} \\ \vdots \\ \mathbf{Y_J} \end{bmatrix}=\begin{bmatrix} \mathbf{X_1} & \mathbf{Z_1} & 0 & 0 & 0 \\ \mathbf{X_2} & 0 & \mathbf{Z_2} & 0 & 0 \\ \vdots & & & \dots & \\ \mathbf{X_J} & 0 & 0 & 0 & \mathbf{Z_J} \end{bmatrix} \begin{bmatrix} \beta \\ b_1 \\ b_2 \\ \vdots \\ b_J \end{bmatrix}+\begin{bmatrix} \epsilon_1 \\ \epsilon_2 \\ \vdots \\ \epsilon_J \end{bmatrix}$$

where $\mathbf{Y_j}$ is a $n_j \times 1$ matrix containing all observations of the response for group $j$, $\mathbf{X_j}$ and $\mathbf{Z_j}$ are $n_j \times 2$ design matrices in this case and $\epsilon_j$ is again a $n_j \times 1$ matrix.

Writing them out, we have:

$\mathbf{Y_j} = \begin{bmatrix} y_{1j}\\ y_{2j}\\ \vdots \\ y_{n_jj} \end{bmatrix}, \mathbf{X_j}=\mathbf{Z_j}=\begin{bmatrix} 1 & x_{1j} \\ 1 & x_{2j} \\ \vdots & \vdots \\ 1 & x_{n_jj} \end{bmatrix}$ and $\epsilon_j = \begin{bmatrix} \epsilon_{1j}\\ \epsilon_{2j}\\ \vdots \\ \epsilon_{n_jj} \end{bmatrix}.$

The regression coefficient vectors then are

$\beta = \begin{pmatrix} \beta_0 \\ \beta_1 \end{pmatrix}$, $b_j=\begin{pmatrix} u_{0j}\\ u_{1j} \end{pmatrix}$

To see that the two model formulations are indeed equivalent, let us look at any of the groups (let's say the $j$-th one).

$$\mathbf{Y_j} = \mathbf{X_j} \beta + \mathbf{Z_j}b_j + \epsilon_j$$

Applying above definitions, one can show that the $i$-th row of the resulting vector is just $$y_{ij}= \beta_0 + \beta_1 x_{ij} + u_{0j}+u_{1j}x_{ij}+\epsilon_{ij},$$ where $i$ ranges from $1$ to $n_j$.

• +1, I would just point out that there are large computational advantages from implementing using $Z_j$ rather than the full $Z$ matrix. The $Z_j$ are basically a sparse matrix storage version of $Z$ Feb 25, 2013 at 8:02