How many fixed factors can be entered in a linear mixed effects model?

Question

My data set has 60 participants. Data was collected from each of them 3 times over seven months, and at each time, each participant did two types of speaking tasks (monologic and dialogic). So my spreadsheet has 360 rows (60 participants x 3 times x 2 tasks each time=360). The dependent variable is proficiency scores (each participant took a proficiency test at each time. So I have 60x3=180 proficiency scores).The fixed factors are the linguistic variables derived from the monologic and dialogic speaking tasks that the participants did at each time. I am building Linear Mixed Effects (LME) models on R to examine:

1. whether the relationship between the linguistic predictors (fixed factors) and the proficiency scores (dependent variable) varied depending on the speaking task-type (monologic/dialogic) and

2. whether the relationship between the linguistic predictors (fixed factors) and the proficiency scores (dependent variable) varied depending on the time (time 1/time 2/time 3).

My question is how many fixed factors can I enter in each LME model? I need to include task, time, and text-length(to control for the confounding effect of the amount of speech produced) as fixed factors in each LME model. But apart from these 3, how many fixed factors can be entered in each LME model? I am aware that in linear regression models, the rule of thumb is 1 predictor per 10 observations. Does this also apply to LME? If yes, am I allowed to include only 6 predictors in each LME model? I would appreciate any help. Thank you very much.

Answer 1

There are no hard rules to follow as to how many fixed effects can enter your model so long that you have sufficient observations to to make your design matrix full rank and therefore, your effects uniquely estimable. To uniquely estimate your effects, you'll at least need one observation for every parameter that will be estimated by your model. Of course, the more observations you have, the more powerful your analysis will be and the easier it will be to detect differences when carrying out statistical tests.

It seems like you've already fixed the sample size, so you'll need to carry out a power analysis for the given size to determine how powerful your tests will actually be once you've decided what your model will look like. There are many resources on the web and papers written on sample size estimation for linear mixed effects models. I've listed a few below. Statistical software will alert you if you have an insufficient number of observations to carry out the analysis so pay attention to any warnings or error messages. You may also be interested in the R package sjstats and the smpsize_lmm function which computes 'an approximated sample size for linear mixed models (two-level-designs), based on power-calculation for standard design and adjusted for design effect for 2-level-designs.'

1. Easy Power and Sample Size for Most of the Mixed Models
2. Power and Sample Size in Multilevel Linear Models
3. An Introduction to Statistical Power Calculations for Linear Models with SAS 9.1
4. An Overview of Variance Inflation Factors for Sample-Size Calculation

Answer 2

As Robert Long pointed out in a comment, the critical thing is whether you evaluated power adequately before you did the study, and your response to his comment indicates that you did. As you have collected the data, you should analyze the results in the way that you intended to when you did the power analysis. If you are worried about overfitting, the usual problem with too many predictors, you could see whether the results on multiple bootstrap samples of the data provide substantially different results than what you got with the full data set. That's a way to estimate the "optimism" in your original model.

Rules of thumb are just that: rules of thumb, attempts to point you to an adequate size of study to allow detection of effects of interest without overfitting. As you asked about this rule of thumb, lets' see how it might apply in practice here.

Your "rule of thumb is 1 predictor per 10 observations." (Emphasis added.) You have 180 observations of your outcome variable, so things are not so dire as you might fear, even based on your rule of thumb.

The number of predictors is essentially the number of degrees of freedom (df) you'll be using up with them, so you have to be careful with categorical predictors: they count as 1 predictor for every level beyond the first. In terms of predictors you have task (1 predictor), time (2 predictors if you treat it as categorical), text length (at least 1, more if you fit it with a spline or similar flexible model), and 1 less than the number of your linguistic predictors. You don't say how many of those there are, let's say 4 total for 3 effective predictors. That's 7 to there.

Your design, however, sounds like your interests are in interactions: "whether the relationship between the linguistic predictors (fixed factors) and the proficiency scores (dependent variable) varied depending on the speaking task-type (monologic/dialogic)" and "... varied depending on the time." In applying that rule of thumb, each interaction term also counts as a predictor. So if you have 4 linguistic predictors (3 df) it has 3 interaction predictors with task and 6 with time. Add those 9 to the previous 7 and you have 16 total predictors to here.

What about the random effects? As this is just a rule of thumb anyway, why not count one for each random effect that you are modeling? So that depends on how many random effects you are modeling: intercepts, slopes with respect to task, slopes with respect to time, slopes with respect to text length...

If you are only modeling the variance of the random intercepts, then you have on the order of 17 predictors to go along with your 180 observations.