When should I *not* permit a fixed effect to vary across levels of a random effect in a mixed effects model?

Question

Given a predicted variable (P), a random effect (R) and a fixed effect (F), one could fit two* mixed effects models (lme4 syntax):

m1 = lmer( P ~ (1|R) + F )
m2 = lmer( P ~ (1+F|R) + F)

As I understand it, the second model is the one that permits the fixed effect to vary across levels of the random effect.

In my research I typically employ mixed effects models to analyze data from experiments conducted across multiple human participants. I model participant as a random effect and experimental manipulations as fixed effects. I think it makes sense a priori to let the degree to which the fixed effects affect performance in the experiment vary across participants. However, I have trouble imagining circumstances under which I should nor permit the fixed effects to vary across levels of a random effect, so my question is:

When should one not permit a fixed effect to vary across levels of a random effect?

Answer 1

I am not an expert in mixed effect modelling, but the question is much easier to answer if it is rephrased in hierarchical regression modelling context. So our observations have two indexes $P_{ij}$ and $F_{ij}$ with index $i$ representing class and $j$ members of the class. The hierarchical models let us fit linear regression, where coefficients vary across classes:

$$Y_{ij}=\beta_{0i}+\beta_{1i}F_{ij}$$

This is our first level regression. The second level regression is done on the first regression coefficients:

\begin{align*} \beta_{0i}&=\gamma_{00}+u_{0i}\\\\ \beta_{1i}&=\gamma_{01}+u_{1i} \end{align*}

when we substitute this in first level regression we get

\begin{align*} Y_{ij}&=(\gamma_{00}+u_{0i})+(\gamma_{01}+u_{1i})F_{ij}\\\\ &=\gamma_{00}+u_{0i}+u_{1i}F_{ij}+\gamma_{01}F_{ij} \end{align*}

Here $\gamma$ are fixed effects and $u$ are random effects. Mixed models estimate $\gamma$ and variances of $u$.

The model I've written down corresponds to lmer syntax

P ~ (1+F|R) + F

Now if we put $\beta_{1i}=\gamma_{01}$ without the random term we get

\begin{align*} Y_{ij}=\gamma_{00}+u_{0i}+\gamma_{01}F_{ij} \end{align*}

which corresponds to lmer syntax

P ~ (1|R) + F

So the question now becomes when can we exclude error term from the second level regression? The canonical answer is that when we are sure that the regressors (here we do not have any, but we can include them, they naturally are constant within classes) in the second level regression fully explain the variance of coefficients across classes.

So in this particular case if coefficient of $F_{ij}$ does not vary, or alternatively the variance of $u_{1i}$ is very small we should entertain idea that we are probably better of with the first model.

Note. I've only gave algebraic explanation, but I think having it in mind it is much easier to think of particular applied example.

Answer 2

You can think of a "Fixed effect" as a "random effect" with a variance component of zero.

So, a simple answer to why you wouldn't let fixed effect to vary, is insufficient evidence for a "large enough" variance component. The evidence should come from both the prior information and the data. This is in line with the basic "occam's razor" principle: don't make your model more complex than it needs to be.

I tend to think of linear mixed models in the following way, write out a multiple regression as follows:

$$Y=X\beta+Zu+e$$

So $X\beta$ is the "fixed" part of the model, $Zu$ is the "random" part and $e$ is the OLS style residual. We have $u\sim N(0,D(\theta))$, for "random effect" variance parameters $\theta$ and $e\sim N(0,\sigma^{2}I)$. This gives the standard results $(Zu+e)\sim N(0,ZD(\theta)Z^{T}+\sigma^{2}I)$, which means we have:

$$Y\sim N(X\beta,ZD(\theta)Z^{T}+\sigma^{2}I)$$

Compare this to the OLS regression (which has $Z=0$) and we get:

$$Y\sim N(X\beta,\sigma^{2}I)$$

So the "random" part of the model can be seen as a way of specifying prior information about the correlation structure of the noise or error component in the model. OLS basically assumes that any one error from the fixed part of the model in one case is useless for predicting any other error, even if we knew the fixed part of the model with certainty. Adding a random effect is basically saying that you think some errors are likely to be useful in predicting other errors.

Answer 3

This is quite an old question with some very good answers, however I think it can benefit from a new answer to address a more pragmatic perspective.

When should one not permit a fixed effect to vary across levels of a random effect ?

I won't address the issues already described in the other answers, instead I will refer to the now-famous, though I would rather say "infamous" paper by Barr et al. (2013) often just referred to as "Keep it maximal".

Barr, D.J., Levy, R., Scheepers, C. and Tily, H.J., 2013. Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3):255-278. https://doi.org/10.1016/j.jml.2012.11.001

In this paper the authors argue that all fixed effects should be allowed to vary across levels of the grouping factors (random intercepts). Their argument is quite compelling - basically that by not allowing them to vary, it is imposing constraints on the model. This is well-described in the other answers. However, there are potentially serious problems with this approach, which are described by Bates el al. (2015):

Bates, D., Kliegl, R., Vasishth, S. and Baayen, H., 2015. Parsimonious mixed models. arXiv preprint arXiv:1506.04967

It is worth noting here that Bates is the primary author of the lme4 package for fitting mixed models in R, which is probably the most widely used package for such models. Bates et al. note that in many real-world applications, the data simply won't support a maximal random effects structure, often because there are insufficient numbers of observations in each cluster for the relevant variables. This can manifest itself in models that fail to converge, or are singular in the random effects. The large number of questions on this site about such models attests to that. They also note that Barr et al. used a relatively simple simulation, with "well-behaved" random effects, as the basis for their paper. Instead Bates et al. suggest the following approach:

We proposed (1) to use PCA to determine the dimensionality of the variance-covariance matrix of the random-effect structure, (2) to initially constrain correlation parameters to zero, especially when an initial attempt to fit a maximal model does not converge, and (3) to drop non-significant variance components and their associated correlation parameters from the model

In the same paper, they also note:

Importantly, failure to converge is not due to defects of the estimation algorithm, but is a straightforward consequence of attempting to fit a model that is too complex to be properly supported by the data.

And:

maximal models are not necessary to protect against anti-conservative conclusions. This protection is fully provided by comprehensive models that are guided by realistic expectations about the complexity that the data can support. In statistics, as elsewhere in science, parsimony is a virtue, not a vice.

Bates et al (2015)

From a more applied perspective, a further consideration that should be made is whether or not, the data generation process, the biological/physical/chemical theory that underlies the data, should guide the analyst towards specifying the random effects structure.