Linear Mixed Effects Model Variances

Consider the following model:

$$$$Y_i = X_i\beta + Z_ib_i + \varepsilon_i,$$$$

where $$b_i \sim N(0, D)$$, and $$\varepsilon_i \sim N(0, R_i(\gamma))$$.

The variance of $$Y_i$$ conditional on $$x_i$$ is given by

$$$$\text{var}(Y_i \mid x_i) = Z_iDZ_i^\top + R_i(\gamma).$$$$

The first term represents the contribution to the induced model for the overall covariance pattern due to among-individual sources of variance and correlation, and the second term represents the contribution due to within-individual sources.

What is the difference between among-individual and within-individual sources? Does anyone have some examples?

• Please elaborate on what the various terms in that equation represent, and perhaps a statement of your model in the form Y_i = .... I think you'll find (at least) two variances in the setup: one or more for the random effects (this is the "among-individual" source of variance), and one for the error variance (this is the "within-individual" source of variance).
– user88719
Commented Dec 30, 2019 at 16:08
• I have edited the question.
– JLee
Commented Dec 30, 2019 at 16:41

The terms within-individual variance and among-individual variance are not commonly found in the mixed effects model literature. It more commonly arises in the ANOVA literature, and rather than "among", the usual term is "between". Total variance is partitioned into that which is attributable to differences within individuals, for example the natural variation that occurs in the measurement of blood pressure of a person during a day, and that which is attributable to differences between (or among) individuals. Some people have generally different blood pressure than others, but each person's blood pressure also varies throughout the day.

A repeated measures ANOVA can be formulated as a mixed effects model. In the case of repeated measures within individuals, a random intercepts model will estimate a variance at the individual level, which will be the variance of $$b$$ in your notation above, while the residual variance is at the measurement level, and is the variance of $$\varepsilon$$ in your model. The former is between (among) and the latter is within.

The following pieces of your model are fixed and known: $$X_i$$ and $$Z_i$$. The vector $$\beta$$ is fixed but unknown. You have two random pieces in your model, $$b_i$$ (I would expect this to be $$b$$, i.e. shared for all $$i$$, and I'm going to use $$b$$ below), and $$\epsilon_i$$. I suspect the setup of your problem also specifies that $$b$$ is independent of the $$\epsilon_i$$.

To see where the variance expression comes from, just plug your model formula in: \begin{align} \text{var}(Y_i|X_i) &= \text{var}(X_i\beta + Z_ib + \epsilon_i)\\ \text{(a)}&=\text{var}(Z_ib + \epsilon_i)\\ \text{(b)}&=\text{var}(Z_ib) + \text{var}(\epsilon_i)\\ \text{(c)}&=Z_iDZ_i^T + R_i(\gamma), \end{align} where (a) is because $$X_i\beta$$ is constant, (b) is because $$b$$ and $$\epsilon_i$$ are independent, and (c) comes from looking at the distributions of $$b$$ and $$\epsilon_i$$.

Putting the "among-individual" and "within-individual" language into context is problem-dependent. For example, mixed-effect models can be used to represent penalized splines for scatterplot smoothing, in which case there are no individuals at all: the random effects are coefficients of basis functions.

For a (totally fictional) example where the language makes sense, suppose an experimenter is modeling sodium concentration $$Y_i$$ in pond water, and they are using potassium concentration as a covariate $$z_i$$. They sample 6 casks (say, 1 gallon each) of water from different locations in the pond, and from each cask, test 5 small 1 milliliter subsamples. We have $$i=1, \dots, 30$$ samples, and each sample belongs to some cask $$k(i) \in \{1, \dots, 6\}$$.

We might model this as a mean, a linear part depending on potassium, a random adjustment to the intercept depending on the cask, and a random measurement error. We could write $$Y_i = [1\,\,\,z_i]^T\beta + Z_i^Tb + \epsilon_i,$$ where $$Z_i$$ is a vector of zeros except for a one at entry $$k(i)$$: it picks out the $$k(i)$$ entry of $$b$$. We assume $$b \sim N(0,\sigma_b^2 I)$$ and $$\epsilon_i \sim N(0,\sigma_\epsilon^2)$$. Our random vector $$b$$ has six entries: one for each cask, and they represent the variability between casks. You'll find that $$\text{var}(Y_i) = \sigma_b^2 + \sigma_\epsilon^2$$, and that $$\text{cov}(Y_i,Y_j) = \sigma_b^2$$ if observations $$i$$ and $$j$$ came from the same cask, but $$\text{cov}(Y_i,Y_j) = 0$$ if they came from different casks.

• I understand the derivations. I was looking for more clarity in what is meant by within-individual and among-individual variances, and within-individual and among-individual covariates and examples of each. More of a way to put the maths into context.
– JLee
Commented Dec 30, 2019 at 17:14