Suppose I have the following two generalized linear mixed models (GLMM): \begin{align*} g(\mathbb{E}[Y_{ij}|X_{ij}]) &= \beta^\intercal_1 X_{ij}^{(1)} + \beta_2^\intercal X_{i}^{(2)} + U_i && \text{Model 1} \\ g(\mathbb{E}[Y_{ij}|X_{ij}]) &= \beta^\intercal_1 X_{ij}^{(1)} + U_i && \text{Model 2} \end{align*} where $X_{ij} = (X^{(1)}_{ij}, X^{(2)}_{i})$ are partitioned into individual-level covariates and cluster-level covariates. If I treat $U_i$ as random effects, there is something to be gained in fitting the first model instead of absorbing the $\beta_2^\intercal X_i^{(2)}$ into $U_i$ because $X_{i}^{(2)}$ may not follow, say, a normal distribution when we normally fit a GLMM.

But, say instead I fitted $U_i$ as a fixed effect (i.e. a factor for each cluster), then what benefits are there to Model 1 over Model 2 when computing the fitted values $\widehat{\mathbb{E}}[Y|X]$ (not new values $\widehat{\mathbb{E}}[Y|X_\text{new}]$, where the new cluster level covariates might not match the existing covariates for that cluster).


Considerations of random effects model with and without cluster level covariates:

  1. As you already said: if the random effects only follow a normal (or whatever other distribution you assume after adjustment for cluster level covariates), you should use them.
  2. Reducing unexplained between cluster variability = more precise inference/less uncertainty.
    1. Risk of overfitting, if you try to take too many covariates into account (for cluster covariates, you really only have as many observations as clusters). If you overfit these covariates, you might end up inappropriately not adjusting for the clustered nature of the model.

Fixed effects with vs. without cluster level covariates:

  1. If these covariates (they cannot be categorical though, otherwise they are not identifiable in a fixed effects model) explain vs variability, they still help. In a way, all the information is already in the cluster fixed effect, but the covariate may present the data more usefully/in a way that makes out easier for the model.
  2. Overfitting is still a concern (may get colinearity or nearly so between covariates and cluster identities).

Fixed vs. random effects

  1. Fewer assumptions are made by the fixed cluster effect model (minimizes bias).
  2. The random effects model is more efficient, if it is approximately right (minimizes variance of estimators) - with less bias introduced the "more right" it is. Cluster level covariates that explain key cluster differences will make it more plausible that the residual between cluster variability might be normal. Misspecifying the functional relationship for cluster covariates may be a concern, but should not matter too much in a linear model and bias inference towards no effect in most other GLMMs.
  3. The random effects model will tend to weight evidence from small clusters more similarly to that from large clusters compared with the fixed effects model. Taking into account important cluster level covariates can reduce this effect.
  • $\begingroup$ A great response, Bjorn. Another point to consider in the fixed vs. random effects question is the number of clusters. Random effects models generally work well with 15-20 clusters, and using appropriate corrections for small sample size (e.g., Kenward-Roger), but much below that, and many recommend using "fixed effects" models with dummy variable indicators for each of the clusters. See Mcneish & Kelley's new paper on this in Psychological Methods. www3.nd.edu/~kkelley/publications/articles/… $\endgroup$ – Erik Ruzek Aug 1 at 21:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.