Rule of Thumb for Mixed Model

Question

Suppose that we have $n$ observations of $(X,Y)$ pair, where $X$ are real (might be vectors), and $Y$ is real. We want to a linear model. One rule of thumb is that the number of learnable parameters excluding the intercept should not exceed $\frac{n}{10}$. It is not that we do not do it if each learnable parameter only has $9$ observations, but it is a way to prevent overfit.

Is there a similar rule of thumb for mixed models? To take a simple example, suppose that $X \in \mathbb{R}^n$ and $Y \in \mathbb{R}^n$ are data, we get a class called patient, which is a factor with say $k$ levels. And we wish to do a random intercept:

lmer(Y ~ X + (1 | patient))

Is there a similar rule of thumb to prevent overfit regarding number of learnable parameters, $n$, $k$ and number of observations for each patient?

Answer 1

The practicality of such a rule of thumb would be questionable at best. Even for ordinary linear models, different text books will provide different rules of thumb, see the excellent discussion here for example.

I don't think a simple (meaningful) rule of thumb can exist for mixed models. Consider crossed random effects, nesting, random slopes, correlation between random effects. All of these will, to varying extents, affect the required sample size. And that is only considering the random structure.

If the variance between groups is large enough in comparison to the error variance, you can simplify your rule of thumb by just counting groups ($k$), and not observations ($n$). With that in mind, some helpful rules of thumb taken from other posts on this site include:

• A minimum number of groups to even estimate a random effect should be about $k \geq 6$.
• As for the number of observations per group, for sufficiently large number of groups ($k \gtrapprox 500$), small group sizes and even single-sample groups are not a major problem.
• For generalized linear models, the minimum number should be substantially higher, especially with binary outcomes, where even a fixed effects model with only an intercept is recommended to have $n > 96$.

And finally, while not a rule of thumb, there are several posts with references to useful packages outlining simulation based approaches are better than rules of thumb [1], [2], [3].