Reconciling notations for mixed models

Question

I am familiar with notation such as:

\begin{align} y_{ij} &= \beta_0 + \beta_i x_{ij} + u_j + e_{ij}\\ &= \beta_{0j} + \beta_i x_{ij} + e_{ij} \end{align} where $\beta_{0j}=\beta_{0}+u_j$, and

\begin{align} y_{ij} &= \beta_0 + \beta_1 x_{ij} + u_{0j} + u_{1j} x_{ij} + e_{ij} \\ &= \beta_{0j} + \beta_{1j} x_{ij} + e_{ij} \end{align} where $\beta_{0j}=\beta_{0}+u_{0j}$ and $\beta_{1j}=\beta_1+u_{1j}$

for a random intercepts model and a random slope + random intercepts model, respectively.

I have also come across this matrix/vector notation, which I have been told is "mixed model notation for grown ups" (according to my elder brother):

$$ \mathbf{y}=\mathbf{X\beta} + \mathbf{Z b} + \mathbf{e} $$ where $\mathbf{\beta}$ are the fixed effects and $\mathbf{b}$ are the random effects.

If I have understood correctly, the latter notation is a more general notation for the former which are specific versions of the latter.

I should like to see how the former can be derived from the latter.

Answer 1

We consider a mixed model with random slopes and random intercepts. Given that we have only one regressor, this model can be written as $$ y_{ij}= \beta_0 + \beta_1 x_{ij} + u_{0j}+u_{1j}x_{ij}+\epsilon_{ij}, $$ where $y_{ij}$ denotes the $i$-th observation of group $j$ of the response, and $x_{ij}$ and $\epsilon_{ij}$ the respective predictor and error term.

This model can be expressed in matrix notation as follows:

$$\mathbf{Y}=\mathbf{X}\beta + \textbf{Zb} + \epsilon,$$ which is equivalent to

$$\mathbf{Y}= \begin{bmatrix} \mathbf{X} & \textbf{Z} \end{bmatrix} \begin{bmatrix} \beta\\ \mathbf{b} \end{bmatrix}+ \epsilon $$

Let us assume that we have $J$ groups, i.e. $j=1,\dots,J$ and let $n_j$ denote the number of observations in the $j$-th group. Partitioned for each group, we can write above formula as

$$\begin{bmatrix} \mathbf{Y_1} \\ \mathbf{Y_2} \\ \vdots \\ \mathbf{Y_J} \end{bmatrix}=\begin{bmatrix} \mathbf{X_1} & \mathbf{Z_1} & 0 & 0 & 0 \\ \mathbf{X_2} & 0 & \mathbf{Z_2} & 0 & 0 \\ \vdots & & & \dots & \\ \mathbf{X_J} & 0 & 0 & 0 & \mathbf{Z_J} \end{bmatrix} \begin{bmatrix} \beta \\ b_1 \\ b_2 \\ \vdots \\ b_J \end{bmatrix}+\begin{bmatrix} \epsilon_1 \\ \epsilon_2 \\ \vdots \\ \epsilon_J \end{bmatrix}$$

where $\mathbf{Y_j}$ is a $n_j \times 1 $ matrix containing all observations of the response for group $j$, $\mathbf{X_j}$ and $\mathbf{Z_j}$ are $n_j \times 2 $ design matrices in this case and $\epsilon_j$ is again a $n_j \times 1 $ matrix.

Writing them out, we have:

$\mathbf{Y_j} = \begin{bmatrix} y_{1j}\\ y_{2j}\\ \vdots \\ y_{n_jj} \end{bmatrix}, \mathbf{X_j}=\mathbf{Z_j}=\begin{bmatrix} 1 & x_{1j} \\ 1 & x_{2j} \\ \vdots & \vdots \\ 1 & x_{n_jj} \end{bmatrix}$ and $\epsilon_j = \begin{bmatrix} \epsilon_{1j}\\ \epsilon_{2j}\\ \vdots \\ \epsilon_{n_jj} \end{bmatrix}.$

The regression coefficient vectors then are

$\beta = \begin{pmatrix} \beta_0 \\ \beta_1 \end{pmatrix}$, $b_j=\begin{pmatrix} u_{0j}\\ u_{1j} \end{pmatrix}$

To see that the two model formulations are indeed equivalent, let us look at any of the groups (let's say the $j$-th one).

$$ \mathbf{Y_j} = \mathbf{X_j} \beta + \mathbf{Z_j}b_j + \epsilon_j$$

Applying above definitions, one can show that the $i$-th row of the resulting vector is just $$ y_{ij}= \beta_0 + \beta_1 x_{ij} + u_{0j}+u_{1j}x_{ij}+\epsilon_{ij}, $$ where $i$ ranges from $1$ to $n_j$.