7
$\begingroup$

I have always heard random effects require a minimum number of levels to be correctly specified in a hierarchical (mixed-effects) model. I can admit to following this rule without question (mostly because I don't have a background in statistics). I recently learned it may not be necessary though, or at least there could be exceptions to the rule. I don't have a specific example in mind, but Gomes (2021) discussed how, for linear models at least, less than 5 levels can be perfectly acceptable for fitting a random effect. Could linear models specifically be robust to this rule?

Source

Gomes DGE. 2022. Should I use fixed effects or random effects when I have fewer than five levels of a grouping factor in a mixed-effects model? PeerJ 10:e12794 https://doi.org/10.7717/peerj.12794

$\endgroup$
2
  • 1
    $\begingroup$ Since it is available on BioRxiv,, you might include a link to biorxiv.org/content/10.1101/2021.04.11.439357v2 $\endgroup$
    – Henry
    Oct 28, 2023 at 16:31
  • 2
    $\begingroup$ Actually better link to the published version in PeerJ, where one can see the reviewers' comments as well: peerj.com/articles/12794. An interesting editorial decision about the title whose final version is "Should I use fixed effects or random effects when I have fewer than five levels of a grouping factor in a mixed-effects model?" $\endgroup$
    – dipetkov
    Oct 28, 2023 at 16:34

1 Answer 1

10
$\begingroup$

Random Effect Levels

The five levels argument (which I believe was originally noted by Gelman & Hill, 2007, p.271) is often used because we want our models to have enough information to begin with, but whether or not we "need" five is dependent on the data. A practical reason to use a random effects model is to reduce a lot of variation in data that you would otherwise not be interested in theoretically, but are aware can influence the overall relationship of your IVs and DVs. For this, it is helpful to have many levels but not entirely necessary depending on what you're doing. Having many levels can also ensure greater stability in estimation (to achieve convergence), but that also isn't immediately a given fact if there are other issues with your random effects (such as imbalances in units, etc.) as noted in Austin & Leckie, 2018 and Clarke, 2008. Finally, if you have something like a two-level factor that you are considering as a random effect, it is often easier to just include it as a fixed effect (indeed this was advice originally given by Gelman and Hill). A good example is if we measure math ability as predicted by IQ, which may vary between two neighborhoods. The neighborhoods here could be considered fixed or random, but if we only have two, its simply easier to contrast their coefficients in a regression if there isn't a lot of variation between them.

To summarize, the five-levels heuristic is a useful one, but its not a law. It is useful to know more about your data before adding random effects in anyway.

References

  • Austin, P. C., & Leckie, G. (2018). The effect of number of clusters and cluster size on statistical power and Type I error rates when testing random effects variance components in multilevel linear and logistic regression models. Journal of Statistical Computation and Simulation, 88(16), 3151–3163. https://doi.org/10.1080/00949655.2018.1504945
  • Clarke, P. (2008). When can group level clustering be ignored? Multilevel models versus single-level models with sparse data. Journal of Epidemiology & Community Health, 62(8), 752–758. https://doi.org/10.1136/jech.2007.060798
  • Gelman, A., & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge University Press.
$\endgroup$
2
  • $\begingroup$ Is there an unintentional mismatch between the references in the text and in the list at the end? $\endgroup$
    – dipetkov
    Oct 28, 2023 at 20:15
  • 1
    $\begingroup$ Good catch. I woke up at 3 AM when I wrote that so I probably wasn't coherent yet :) I've fixed that accordingly $\endgroup$ Oct 28, 2023 at 20:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.