When should I *not* permit a fixed effect to vary across levels of a random effect in a mixed effects model?

Question

Given a predicted variable (P), a random effect (R) and a fixed effect (F), one could fit two* mixed effects models (lme4 syntax):

m1 = lmer( P ~ (1|R) + F )
m2 = lmer( P ~ (1+F|R) + F)

As I understand it, the second model is the one that permits the fixed effect to vary across levels of the random effect.

In my research I typically employ mixed effects models to analyze data from experiments conducted across multiple human participants. I model participant as a random effect and experimental manipulations as fixed effects. I think it makes sense a priori to let the degree to which the fixed effects affect performance in the experiment vary across participants. However, I have trouble imagining circumstances under which I should nor permit the fixed effects to vary across levels of a random effect, so my question is:

When should one not permit a fixed effect to vary across levels of a random effect?

Answer 1

I am not an expert in mixed effect modelling, but the question is much easier to answer if it is rephrased in hierarchical regression modelling context. So our observations have two indexes $P_{ij}$ and $F_{ij}$ with index $i$ representing class and $j$ members of the class. The hierarchical models let us fit linear regression, where coefficients vary across classes:

$$Y_{ij}=\beta_{0i}+\beta_{1i}F_{ij}$$

This is our first level regression. The second level regression is done on the first regression coefficients:

\begin{align*} \beta_{0i}&=\gamma_{00}+u_{0i}\\ \beta_{1i}&=\gamma_{01}+u_{1i} \end{align*}

when we substitute this in first level regression we get

\begin{align*} Y_{ij}&=(\gamma_0+u_{0i})+(\gamma_{01}+u_{1i})F_{ij}\\ &=\gamma_0+u_{0i}+u_{1i}F_{ij}+\gamma_{01}F_{ij} \end{align*}

Here $\gamma$ are fixed effects and $u$ are random effects. Mixed model estimates $\gamma$ and variances of $u$.

The model I've written down corresponds to lmer syntax

P ~ (1+F|R) + F

Now if we put $\beta_{1i}=\gamma_{01}$ without the random term we get

\begin{align*} Y_{ij}=\gamma_0+u_{0i}+\gamma_{01}F_{ij} \end{align*}

which corresponds to lmer syntax

P ~ (1|R) + F

So the question now becomes when can we exclude error term from the second level regression? The canonical answer is that when we are sure that the regressors (here we do not have any, but we can include them, they naturally are constant within classes) in the second level regression fully explain the variance of coefficients across classes.

So in this particular case if coefficient of $F_{ij}$ does not vary, or alternatively the variance of $u_{1i}$ is very small we should entertain idea that we are probably better of with the first model.

Note. I've only gave algebraic explanation, but I think having it in mind it is much easier to think of particular applied example.

Answer 2

You can think of a "Fixed effect" as a "random effect" with a variance component of zero.

So, a simple answer to why you wouldn't let fixed effect to vary, is insufficient evidence for a "large enough" variance component. The evidence should come from both the prior information and the data. This is in line with the basic "occam's razor" principle: don't make your model more complex than it needs to be.

I tend to think of linear mixed models in the following way, write out a multiple regression as follows:

$$Y=X\beta+Zu+e$$

So $X\beta$ is the "fixed" part of the model, $Zu$ is the "random" part and $e$ is the OLS style residual. We have $u\sim N(0,D(\theta))$, for "random effect" variance parameters $\theta$ and $e\sim N(0,\sigma^{2}I)$. This gives the standard results $(Zu+e)\sim N(0,ZD(\theta)Z^{T}+\sigma^{2}I)$, which means we have:

$$Y\sim N(X\beta,ZD(\theta)Z^{T}+\sigma^{2}I)$$

Compare this to the OLS regression (which has $Z=0$) and we get:

$$Y\sim N(X\beta,\sigma^{2}I)$$

So the "random" part of the model can be seen as a way of specifying prior information about the correlation structure of the noise or error component in the model. OLS basically assumes that any one error from the fixed part of the model in one case is useless for predicting any other error, even if we knew the fixed part of the model with certainty. Adding a random effect is basically saying that you think some errors are likely to be useful in predicting other errors.