When to use mixed effect model?

Question

Linear Mixed Effects Models are Extensions of Linear Regression models for data that are collected and summarized in groups. The key advantages is the coefficients can vary with respect to one or more group variables.

However, I am struggling with when to use mixed effect model? I will elaborate my questions by using a toy example with extreme cases.

Let's assume we want to model height and weight for animals and we use species as grouping variable.

• If different group / species are really different. Say a dog and elephant. I think there is no point of using mixed effect model, we should build a model for each group.

• If different group / species are really similar. Say a female dog and a male dog. I think we may want use gender as a categorical variable in the model.

So, I assume we should use mixed effect model in the middle cases? Say, the group are cat, dog, rabbit, they are similar sized animals but different.

Is there any formal argument to suggest when to use mixed effect model, i.e., how to draw lines among

1. Building models for each group
2. Mixed effect model
3. Use group as a categorical variable in regression

My attempt: Method 1 is the most "complex model" / less degree of freedom and method 3 is the most "simple model" / more degree of freedom. And Mixed effect model is in the middle. We may consider how much data and how complicated data we have to select the right model according to Bais Variance Trade Off.

Answer 1

I'm afraid I might have the nuanced and perhaps unsatisfying answer that it is a subjective choice by the researcher or data analyst. As mentioned elsewhere in this thread, it isn't enough to simply say the data have a "nested structure." To be fair, though, this is how many books describe when to use multilevel models. For example, I just pulled Joop Hox's book Multilevel Analysis off of my bookshelf, which gives this definition:

A multilevel problem concerns a population with a hierarchical structure.

Even in a pretty good textbook, the initial definition seems to be circular. I think this is partially due to the subjectivity of determining when to use what kind of model (including a multilevel model).

Another book, West, Welch, & Galecki's Linear Mixed Models says these models are for:

outcome variables in which the residuals are normally distributed but may not be independent or have constant variance. Study designs leading to data sets that may be appropriately analyzed using LMMs include (1) studies with clustered data, such as students in classrooms, or experimental designs with random blocks, such as batches of raw material for an industrial process, and (2) longitudinal or repeated-measures studies, in which subjects are measured repeatedly over time or under different conditions.

Finch, Bolin, & Kelley's Multilevel Modeling in R also talks about violating the iid assumption and correlated residuals:

Of particular importance in the context of multilevel modeling is the assumption [in standard regression] of independently distributed error terms for the individual observations within a sample. This assumption essentially means that there are no relationships among individuals in the sample for the dependent variable once the independent variables in the analysis are accounted for.

I believe that a multilevel model makes sense when there is reason to believe that observations are not necessarily independent of one another. Whatever "cluster" accounts for this non-independence can be modeled.

An obvious example would be children in classrooms—they are all interacting with one another, which might lead their test scores to be non-independent. What if one classroom has someone that asks a question that leads to material being covered in that class that isn't covered in other classes? What if the teacher is more awake for some classes than others? In this case, there would be some non-independence of data; in multilevel words, we could expect some variance in the dependent variable to be due to the cluster (i.e., class).

Your example of a dog versus an elephant depends on the independent and dependent variables of interest, I think. For example, let's say we are asking if there is an effect of caffeine on activity level. Animals from all over the zoo are randomly assigned to either get a caffeinated drink or a control drink.

If we are a researcher that is interested in caffeine, we might specify a multilevel model, because we really care about the effect of caffeine. This model would be specified as:

activity ~ condition + (1+condition|species)

This is particularly helpful if there are a large number of species we are testing this hypothesis over. However, a researcher might be interested in the species-specific effects of caffeine. In that case, they could specify species as a fixed effect:

activity ~ condition + species + condition*species

This obviously is a problem if there are, say, 30 species, creating an unwieldy 2 x 30 design. However, you can get pretty creative with how one models these relationships.

For example, some researchers are arguing for an even wider use of multilevel modeling. Gelman, Hill, & Yajima (2012) argue that multilevel modeling could be used as a correction for multiple comparisons—even in experimental research where the structure of the data is not obviously hierarchical in nature:

Harder problems arise when modeling multiple comparisons that have more structure. For example, suppose we have five outcome measures, three varieties of treatments, and subgroups classified by two sexes and four racial groups. We would not want to model this 2 × 3 × 4 × 5 structure as 120 exchangeable groups. Even in these more complex situations, we think multilevel modeling should and will eventually take the place of classical multiple comparisons procedures.

Problems can be modeled in various ways, and in ambiguous cases, multiple approaches might seem appealing. I think our job is to choose a reasonable, informed approach and do so transparently.

Answer 2

In mixed effects models, you add random (error) terms to your model, so you "mix" fixed and random effects. So, another approach to consider when to use mixed effects models, might be to look at what a "random effect" is. Thus, in addition to the previously given answers, I also find the distinction between the terms "fixed" and "random" effects from Bates (2010) instructive, section 1.1 (esp. page 2).

Parameters associated with the particular levels of a covariate are sometimes called the “effects” of the levels. If the set of possible levels of the covariate is fixed and reproducible we model the covariate using fixed-effects parameters. If the levels that we observed represent a random sample from the set of all possible levels we incorporate random effects in the model. There are two things to notice about this distinction between fixed-effects parameters and random effects. First, the names are misleading because the distinction between fixed and random is more a property of the levels of the categorical covariate than a property of the effects associated with them.

This definition often applies to some hierachical structure like countries, or classrooms, because you always have a "random" sample of countries or classrooms - data has not been collected from all possible countries or classrooms.

Sex, however, is fixed (or at least treated as being fixed). If you have male or female persons, there are no other sex-levels left (there might be some gender-exceptions, but this is mostly ignored).

Or say educational level: If you ask whether people are of lower, middle or higher education, there are no levels left, so you have not taken a "random" sample of all possible educational levels (hence, this is a fixed effect).

Answer 3

You could of course build a model for each different group, there is nothing wrong with that. However, you'd require larger sample size and need to manage multiple models.

By using mixed model, you pool (and share) the data together and thus require smaller sample size.

In doing so, we are sharing statistical strength. The idea here is that something we can infer well in one group of data can help us with something we cannot infer well in another.

Mixed models also prevents over-sampled groups from unfairly dominating inference.

My point is if you want to model the underlying latern hierarchical structure, you should add random effects to your model. Otherwise, if you don't care in your model intrepretation you don't use it.

https://www.dropbox.com/s/rzi2rsou6h817zz/Datascience%20Presentation.pdf?dl=0

gives relevant discussion. The author discussed why he didn't want to run separate regression models.

Answer 4

You use mixed models when some reasonable assumptions can be made, based on the study design, about the nature of correlation between observations and inference is desired on individual level or conditional effects. Mixed models allow for specifications of random effects, which are a convenient representation of correlation structures that arise naturally in the collection of data.

The most common type of mixed model is a random intercepts model which estimates a latent distribution of common constants having a 0-mean, finite variance normal distribution within clusters of individuals identified in the dataset. This approach accounts for potentially hundreds of confounding factors common to groups of observations, or clusters, but varying between clusters.

A second common type of mixed model is a random slopes model which, akin to the random intercepts model, estimates a latent distribution of time-predictor interactions which again comes from a 0-mean, finite variance normal distribution within a panel study, or clusters of observations measured prospectively or in a longitudinal fashion.

These results are roughly similar to the results obtained from using generalized least squares and the EM-algorithm to iteratively estimate model parameters and the covariance between these dependent observations (or more precisely, their residuals). Weighted least squares is more efficient than least squares when the covariance between observations is known. While the covariance is rarely known, it can be assumed to take a particular structure and be estimated iteratively. The random intercepts model gives similar inference and likelihoods to a weighted least squares having an exchangeable correlation structure where $cor(Y_1, Y_2) = \rho$ if $Y_1, Y_2$ are in the same cluster, and 0 otherwise. The random slopes model gives similar inference and likelihoods to a weighted least squares having an autoregressive 1 correlation structure where $cor(Y_t, Y_s) = \rho^{|t-s|}$ if $Y_t, Y_s$ are observations on the same sample at different times $t, s$ and 0 otherwise. The results are not identical, because the random intercept forces observations within clusters to be positively associated which is almost always a reasonable assumption.

Individual level or conditional effects can be contrasted with population level or marginal effects. Marginal effects represent the effect in a population from an intervention or screening. As an example, an intervention to increase compliance in substance abuse rehabilitation may look at attendance over 3 months in a panel of patients admitted for various conditions. Duration of usage may vary between patients and strongly predict compliance with the workshop with longer using participants having greater addictive tendencies and avoidance. An individual level analysis may reveal that the study is effective despite the fact that participants with longer addiction did not attend prior to receiving the intervention and continued not to attend after receiving the intervention. The inference may be problematic if in the population most eligible people have a long duration of addiction.

Marginal effects has less precise inference due to ignoring homogeneity between clusters in time or space. They can be estimated with generalized estimating equations or by marginalizing the mixed models.

Answer 5

Mixed-effects should be used when data have a nested or hierarchical structure. This actually violates the assumption of independence of measurements, because all measurements within the same group/level are correlated. In case of

"If different group/species are really similar. Say a female dog and a male dog. I think we may want use gender as a categorical variable in the model."

gender would be factor variable and fixed-effect, whereas variability of dog sizes within gender is a random-effect. My model would be

response ~ sex + (1|size), data=data

Intuitively, rabits, dogs and cates should be modeled separately as sizes of dog and cat are not correlated, however size of two dogs is a kind of "within-species" variability.