In regression, can an effect be random if all group-level population units are sampled (but not all subsamples)?

Question

I'm not clear on why we use random effects. I see two cases: Either we do it because we want to model the heterogeneity of cluster or group-level effects within a population sample. Or, I see it chosen out convenience: Because we don't care about individual cluster or group-level effects, and thus using random effects and only modeling a single coefficient for the random effect is more parsimonious and therefore leads to a better model in principle.

For example, in Introduction to Regression, Montgomery says:

The units used for a specific random effect represent a random sample from a much larger population of possible units. For example, patients in a biomedical study often are random effects. The analyst selects the patients for the study from a large population of possible people. The focus of all statistical inference is not on the specific patients selected; rather, the focus is on the population of all possible patients. The key point underlying all random effects is this focus on the population and not on the specific units selected for the study.

However, I also see random effects simply described in terms of principles of simplicity:

If interest lies in mean intercept and slope $(\alpha, \beta)$ and sex difference $δ$s but not individual subjects then wasteful to include subject specific fixed effects $a_i$ and $b_i$.

So my question is: Is it appropriate to consider an effect "random" if we have included all group-level factors (all groups)? In this case, we would just be modeling the heterogeneity of the groups as defined by the subsamples and their sampling error, rather than the sampling error of the groups themselves, if that makes sense. Is this still a "random" effect?

Answer 1

I have discussed the importance of fixed vs random effects multiple times here, including this post and this post. However, it appears there are some specifics to your question that may be important to discuss.

Either we do it because we want to model the heterogeneity of cluster or group-level effects within a population sample. Or, I see it chosen out convenience: Because we don't care about individual cluster or group-level effects, and thus using random effects and only modeling a single coefficient for the random effect is more parsimonious and therefore leads to a better model in principle.

To the latter point, it is not smart to simply apply a mixed effects model without consideration of why the random effects structure is specified (see McNeish et al., 2017). While models are commonly specified willy-nilly, there really should be a reason, otherwise it can be quite overkill. If we know we can simply model a two-factor predictor as a fixed effect in the model, for example, we really don't need a mixed models design. The clarity and beauty of mixed models is describing a large level of variance in clustering. The first point you made is closer to the truth: that we know that for the given regression

$$ y = X\beta + \epsilon $$

the $y$ and $\epsilon$ terms are random, and knowing how to decompose some of the $\epsilon$ part can be helpful to knowing how much our fixed effects are actually associated with the outcome. So the first quoted section you provided matches that expectation.

The second quoted section somewhat gets to what I noted about when it is necessary to include such random effects. Sometimes patients will barely vary at all. If we know that generally the mean of $y$ is generally $50$ for all subjects, and their means generally don't deviate from that, then a random effects model is not necessary (particularly if it also includes random slopes, which can really cause models to quickly crash).

For your final question:

So my question is: Is it appropriate to consider an effect "random" if we have included all group-level factors (all groups)?

This simply depends on your data and hypothesized relationships (see the links I have added in my answer to my previous explanations of how to differentiate this). When this becomes particularly problematic in my opinion is when you have something like 20 groups all modeled into the same regression. Sure, you can see what adjustments to the conditional mean are made based off each group, but it quickly becomes difficult to meaningfully interpret. If this isn't the primary reason for your study, then a random effects model may be better. Some additional guidance on this matter can be found in some of the references below.

References

• Harrison, X. A., Donaldson, L., Correa-Cano, M. E., Evans, J., Fisher, D. N., Goodwin, C. E. D., Robinson, B. S., Hodgson, D. J., & Inger, R. (2018). A brief introduction to mixed effects modelling and multi-model inference in ecology. PeerJ, 6, e4794. https://doi.org/10.7717/peerj.4794
• McNeish, D., Stapleton, L. M., & Silverman, R. D. (2017). On the unnecessary ubiquity of hierarchical linear modeling. Psychological Methods, 22(1), 114–140. https://doi.org/10.1037/met0000078
• Meteyard, L., & Davies, R. A. I. (2020). Best practice guidance for linear mixed-effects models in psychological science. Journal of Memory and Language, 112, 104092. https://doi.org/10.1016/j.jml.2020.104092