Is certain data better for Fixed effects compared to Mixed effects?

Question

I think it is an obvious question with an obvious answer, but I want to confirm.

• In hierarchical regression models, if some clusters have very few observations, the coefficients for those clusters should be less reliable compared to clusters with more observations.
• In a longitudinal regression model, if some medical patients have very few repeated measures...and other medical patients have more repeated measures...the coefficient estimates for medical patients with fewer measurements should be less reliable compared to patients with more observations.

Therefore, I feel there is an intrinsic relationship here. Some models provide less information but can be more accurate (ex: pure fixed effects models), some models provide more information but can less accurate (ex: mixed effects models). By less accurate, I mean that maybe variance of coefficients can be overestimated or underestimated.

Therefore, certain datasets (ex: where some clusters have a very small population, where some patients have few numbers of measurements) are better suited for fixed effects regression compared to mixed effects regression. Is this a correct interpretation?

PS: I wish there was an equation/flowchart to show if your data can support mixed effects models...or if your data can only support fixed effects models. Datasets with not underpopulated clusters and not patients with many measurements can support both mixed effects and fixed effects...but datasets with underpopulated clusters and patients with many measurements can only support fixed effects. I wish there was a warning sign, ex: data not good enough for mixed effect models, proceed with caution! Use fixed effects!

PS: Are there mathematical references that show why empty clusters and few measurements in repeated measures cause problems? I can't understand references, but I can give to classmates with a better math background.

Answer 1

Fixed vs Random

I have spoken on this topic a decent amount here, so you can probably go through some of my answers and search for "mixed models" and find a decent amount of discussion on this. Here are some important reasons for why a random effect may be helpful:

• Your errors are correlated, therefore biasing the coefficients of the fixed effects.
• The are several clusters, which input into a regression as a factor, will make the regression very difficult to interpret.
• There are complex associations between parts of your data (e.g. subjects repeatedly measured on IQ within classrooms of varying levels of academic ability, which may be related in some way).

There are also reasons not to use a variable as a random effect:

• There is no seeming need to aggregate a lot of levels of a variable. With two levels, it is often sensible to simply include them directly in a regression and will be easy to interpret.
• There is simply not enough variation in your levels to meaningfully include it. If for example many subjects are measured on IQ but don't meaningfully differ from each other much, then this can often cause a mixed model to crash and burn, especially if the models are way too complex to estimate.
• You have a data generating mechanism that really doesn't require partialing out random noise. For example, a bivariate exponential relationship that is measured with very little error probably doesn't have any leftover error variance to even include random effects.
• As you already mentioned, the structure of your random effects (e.g. imbalances in clusters) can also weight this decision.

PS Questions

For the following questions:

I wish there was a equation/flowchart to show if your data can support mixed effect model...or if your data can only support fixed effect model.

There isn't really a super helpful flowchart I have found. However, I would recommend reading through the two references I provide at the bottom of this answer, which discuss best practices and also when mixed models aren't all that useful. I find them to be useful reads and wish I had read them when first introduced to mixed models, as some teachers of this method make some wild assumptions about when you can use them. I also think that this episode and this episode of Quantitude do a good job of discussion on mixed models in general if you want a better sense of things.

Are there mathematical references that show why empty clusters and few measurements in repeated measures cause problems?

I actually provide some references here that include simulation studies and their results regarding this topic. You do not need any sophisticated math background to understand the results.

References

• McNeish, D., Stapleton, L. M., & Silverman, R. D. (2016). On the unnecessary ubiquity of hierarchical linear modeling. Psychological Methods, 22(1), 114–140. https://doi.org/10.1037/met0000078
• Meteyard, L., & Davies, R. A. I. (2020). Best practice guidance for linear mixed-effects models in psychological science. Journal of Memory and Language, 112, 104092. https://doi.org/10.1016/j.jml.2020.104092

Edit

You've asked a lot of questions. FYI its good practice to simply edit additional questions into the original post rather than comment a bunch in the answers...otherwise they get buried and nobody sees them. Questions should generally stick to the original post and additional unrelated questions should be reserved for other posts.

I post this question on ols vs mle ... u have any idea about this one?

This question has nothing to do with this query, so I won't address it. Feel free to edit your question to resolve the original close reasons, otherwise you can ask the question in a way that doesnt closely mimic the original duplicate question.

I just cant understand this point.... if i have 3 repeated measurements for each subject ... in mixed effect, I can estimate a parameter for each subject. it just dosent make sense how this can be a meaningful estimate? somewhere in the math, I will be taking a mean of 3 obs and a variance of 3 obs ... it just seems too little to be considered reliable, yes?

I don't see where this is so unbelievable. Consider if somebody takes a 200 item reading test in Grade 1, then Grade 3, then Grade 5. This allows a lot of room for the response to vary. On average, students may be close to floor at Grade 1, somewhere in the middle for Grade 3, and then be close to ceiling for Grade 5. The level of improvement between grades will vary substantially between people of different abilities. With this in mind, consider if we have Subject 1's reading scores:

• Grade 1: 10/200
• Grade 2: 100/200
• Grade 3: 190/200

Then the subject's scores will have a mean of $\bar{y}=100$ vary by about $\text{SD} = 90$ (or a proportional equivalent). If we then have another subject, Subject 2, whose scores are

• Grade 1: 180/200
• Grade 2: 185/200
• Grade 3: 190/200

Then their reading score mean is $\bar{y} = 185$ and $SD = 5$. This will definitely affect the random intercepts and slopes if we have many subjects who have mean/variance scores in between. The conditional mean of Subject 2 will be higher (the random intercept) and the magnitude of the effect between the predictor and outcome will likely change less for Subject 2 (the random slope). Whether this is wise or important to do depends on your data and the reasons you are modeling it this way.

I will look at links you provided ... but can it mathematically showed that fixed effect model is more stable (in term of properties like unbiased, consistent, sensitivity to outliers, minimum sample size for normality) compared to mixed effects ... in presence of difficult data conditions (ex: underpopulated cluster, few repeated measure)?

Some of this is discussed in the articles I referenced already. While I said the math is not that complex, they do present the mathematical underpinnings of mixed models, so I advise reading through them thoroughly.