Sign up ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

In simple terms, how would you explain (perhaps with simple examples) the difference between fixed effect, random effect and mixed effect models?

share|improve this question
I also find that sometimes is difficult to determine when an effect must be considered as fixed or as random effect. Althought there are some recommendations about this fact, not always is easy to take the right decision. –  Manuel Ramón Nov 19 '10 at 10:29
I think that this link may be helpful in clarifying the underlying principles of mixed models: <a href="…, Random, and Mixed Models (SAS documentation)</a> –  pietrop Sep 4 '13 at 10:27
An extremely helpful answer can also be found here: What is a difference between random effects-, mixed effects- & marginal model? –  gung Nov 19 '14 at 20:21

6 Answers 6

up vote 34 down vote accepted

Blogger Andrew Gelman says that the terms 'fixed effect' and 'random effect' have variable meanings depending on who uses them (link). Perhaps you can pick out which one of the 5 definitions applies to your case. In general it may be better to either look for equations which describe the probability model the authors are using (when reading) or write out the full probability model you want to use (when writing).

share|improve this answer
+1: very nice link! I guess the definition also varies depending on the field (e.g. #4 is very mathematical/statistical, but #1 and #2 are more "understandable" from a life science point of view) –  nico Nov 19 '10 at 6:39
Thanks - this is really helpful. –  Andrew Nov 19 '10 at 19:52
It is also informative to read the Discussion and Rejoinder to this paper. In the discussion, Peter McCullagh wrote that he disagrees with a substantial portion of what Gelman wrote. My point is not to favor one or the other, but to note that there is substantial disagreement among experts and not to put too much weight on one paper. –  julieth Jul 22 '12 at 1:19
Cool, I haven't seen that. Do you have a link to the paper(s) you're talking about? –  John Salvatier Jul 22 '12 at 6:06
The entire discussion is at link –  julieth Jul 22 '12 at 13:34

Fixed effect: Something the experimenter directly manipulates and is often repeatable, e.g., drug administration - one group gets drug, one group gets placebo.

Random effect: Source of random variation / experimental units e.g., individuals drawn (at random) from a population for a clinical trial. Random effects estimates the variability

Mixed effect: Includes both, the fixed effect in these cases are estimating the population level coefficients, while the random effects can account for individual differences in response to an effect, e.g., each person receives both the drug and placebo on different occasions, the fixed effect estimates the effect of drug, the random effects terms would allow for each person to respond to the drug differently.

General categories of mixed effects - repeated measures, longitudinal, hierarchical, split-plot.

share|improve this answer
Your not wrong, but your definition of what a fixed effect is is not what I would think of when someone says fixed effect. Here is what I think of when someone says fixed effect , or this (particularly equation 3 on the Stata page) –  Andy W Nov 19 '10 at 13:44

The distinction is only meaningful in the context of non-Bayesian statistics. In Bayesian statistics, all model parameters are "random".

share|improve this answer
Interesting. But since fixed or random can be considered a condition of a given variable (a given column of data) rather than of a parameter associated with that variable,...does your answer fully apply? –  rolando2 Jan 27 '12 at 0:48

There are good books on this such as Gelman and Hill. What follows is essentially a summary of their perspective.

First of all, you should not get too caught up in the terminology. In statistics, jargon should never be used as a substitute for a mathematical understanding of the models themselves. That is especially true for random and mixed effects models. "Mixed" just means the model has both fixed and random effects, so let's focus on the difference between fixed and random.

Mathematically, the difference between fixed and random effects modeling is that the latter uses a multilevel approach to estimate the variation in a response across multiple groups of observations.* In practice this means that, if you have few data points in a group, the group's effect estimate will be based partially on the more abundant data from other groups. This is known as partial pooling, and it's a nice compromise between completely pooling all groups, which could mask group-level variation, and treating all groups completely separately, which could give poor estimates for low-sample groups.

An example should help prime your intuition. Suppose you want to estimate average US household income by ZIP code. You have a large dataset containing observations of households' incomes and ZIP codes. Some ZIP codes are well represented in the dataset, but others have only a couple households.

For your initial model you would most likely take the mean income in each ZIP. This will work well when you have lots of data for a ZIP, but the estimates for your poorly sampled ZIPs will suffer from high variance. You can mitigate this by using a shrinkage estimator (aka partial pooling), which will push extreme values towards the mean income across all ZIP codes.

But how much shrinkage/pooling should you do for a particular ZIP? Intuitively, it should depend on the following:

  1. How many observations you have in that ZIP
  2. How many observations you have overall
  3. The individual-level mean and variance of household income across all ZIP codes
  4. The group-level variance in mean household income across all ZIP codes

If you model ZIP code as a random effect, the mean income estimate in all ZIP codes will be subjected to a statistically well-founded shrinkage, taking into account all the factors above.

The best part is that random and mixed effects models automatically handle (4), the variability estimation, for all random effects in the model. This is harder than it seems at first glance: you could try the variance of the sample mean for each ZIP, but this will be biased high, because low-sample ZIPs will contribute spurious variance. Imagine sampling one person from a Seattle ZIP - you might get Bill Gates, and if you do, now the mean income for that ZIP is a huge outlier, which pushes the group-level variance estimate way up. In a random effects model, this bias is reduced because the model and inference process naturally account for sampling error and distinguish it from genuine group variation.

Having accounted for (1)-(4), a random/mixed effects model is able to determine the appropriate shrinkage for low-sample groups. It can also handle much more complicated models with many different predictors.

If this sounds like hierarchical Bayesian modeling to you, you're right - it is a close relative but not identical. Mixed effects models are hierarchical in that they have distributions for underlying parameters, but they are typically not fully Bayesian because the top-level hyperparameters will not be given proper priors. For example, in the above example we would most likely treat the mean income in a given ZIP as a sample from a normal distribution, with unknown mean and sigma to be estimated by the mixed-effects fitting process. However, a (non-Bayesian) mixed effects model will typically not have a prior on the unknown mean and sigma, so it's not fully Bayesian. That said, with a decent-sized data set, the standard mixed effects model and the fully Bayesian variant will often give very similar results.

*While many treatments of this topic focus on a narrow definition of "group", the concept is in fact very flexible: it is just a set of observations that share a common property. A group could be composed of multiple observations of a single person, or multiple people in a school, or multiple schools in a district, or multiple varieties of a single kind of fruit, or multiple kinds of vegetable from the same harvest, or multiple harvests of the same kind of vegetable, etc. Any categorical variable can be used as a grouping variable.

share|improve this answer

Not really a formal definition, but I like the following slides: Mixed models and why sociolinguists should use them (mirror), from Daniel Ezra Johnson. A brief recap' is offered on slide 4. Although it mostly focused on psycholinguistic studies, it is very useful as a first step.

share|improve this answer
I think I'm going to need to see that presentation in person to get the full impact. –  Andy W Nov 19 '10 at 13:36

I have written about this in a book chapter on mixed models (chapter 13 in Fox, Negrete-Yankelevich, and Sosa 2004); the relevant pages (pp. 311-315) are available on Google Books. I think the question reduces to "what are the definitions of fixed and random effects?" (a "mixed model" is just a model that contains both). My discussion says a bit less about their formal definition (for which I would defer to the Gelman paper linked by @JohnSalvatier's answer above) and more about their practical properties and utility. Here are some excerpts:

The traditional view of random effects is as a way to do correct statistical tests when some observations are correlated.

We can also think of random effects as a way to combine information from different levels within a grouping variable.

Random effects are especially useful when we have (1) lots of levels (e.g., many species or blocks), (2) relatively little data on each level (although we need multiple samples from most of the levels), and (3) uneven sampling across levels (box 13.1).

Frequentists and Bayesians define random effects somewhat differently, which affects the way they use them. Frequentists define random effects as categorical variables whose levels are chosen at random from a larger population, e.g., species chosen at random from a list of endemic species. Bayesians define random effects as sets of variables whose parameters are [all] drawn from [the same] distribution. The frequentist definition is philosophically coherent, and you will encounter researchers (including reviewers and supervisors) who insist on it, but it can be practically problematic. For example, it implies that you can’t use species as random effect when you have observed all of the species at your field site—since the list of species is not a sample from a larger population—or use year as a random effect, since researchers rarely run an experiment in randomly sampled years—they usually use either a series of consecutive years, or the haphazard set of years when they could get into the field.

Random effects can also be described as predictor variables where you are interested in making inferences about the distribution of values (i.e., the variance among the values of the response at different levels) rather than in testing the differences of values between particular levels.

People sometimes say that random effects are “factors that you aren’t interested in.” This is not always true. While it is often the case in ecological experiments (where variation among sites is usually just a nuisance), it is sometimes of great interest, for example in evolutionary studies where the variation among genotypes is the raw material for natural selection, or in demographic studies where among-year variation lowers long-term growth rates. In some cases fixed effects are also used to control for uninteresting variation, e.g., using mass as a covariate to control for effects of body size.

You will also hear that “you can’t say anything about the (predicted) value of a conditional mode.” This is not true either—you can’t formally test a null hypothesis that the value is equal to zero, or that the values of two different levels are equal, but it is still perfectly sensible to look at the predicted value, and even to compute a standard error of the predicted value (e.g., see the error bars around the conditional modes in figure 13.1).

The Bayesian framework has a simpler definition of random effects. Under a Bayesian approach, a fixed effect is one where we estimate each parameter (e.g., the mean for each species within a genus) independently (with independently specified priors), while for a random effect the parameters for each level are modeled as being drawn from a distribution (usually Normal); in standard statistical notation, $\textrm{species_mean} \sim \textrm{Normal}(\textrm{genus_mean}, \sigma^2_{\textrm{species}}$ ).

I said above that random effects are most useful when the grouping variable has many measured levels. Conversely, random effects are generally ineffective when the grouping variable has too few levels. You usually can’t use random effects when the grouping variable has fewer than five levels, and random effects variance estimates are unstable with fewer than eight levels, because you are trying to estimate a variance from a very small sample.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.