In simple terms, how would you explain (perhaps with simple examples) the difference between fixed effect, random effect and mixed effect models?
Blogger Andrew Gelman says that the terms 'fixed effect' and 'random effect' have variable meanings depending on who uses them (link). Perhaps you can pick out which one of the 5 definitions applies to your case. In general it may be better to either look for equations which describe the probability model the authors are using (when reading) or write out the full probability model you want to use (when writing).
Fixed effect: Something the experimenter directly manipulates and is often repeatable, e.g., drug administration - one group gets drug, one group gets placebo.
Random effect: Source of random variation / experimental units e.g., individuals drawn (at random) from a population for a clinical trial. Random effects estimates the variability
Mixed effect: Includes both, the fixed effect in these cases are estimating the population level coefficients, while the random effects can account for individual differences in response to an effect, e.g., each person receives both the drug and placebo on different occasions, the fixed effect estimates the effect of drug, the random effects terms would allow for each person to respond to the drug differently.
General categories of mixed effects - repeated measures, longitudinal, hierarchical, split-plot.
The distinction is only meaningful in the context of non-Bayesian statistics. In Bayesian statistics, all model parameters are "random".
Not really a formal definition, but I like the following slides: Mixed models and why sociolinguists should use them (mirror), from Daniel Ezra Johnson. A brief recap' is offered on slide 4. Although it mostly focused on psycholinguistic studies, it is very useful as a first step.
There are good books on this such as Gelman and Hill. What follows is essentially a summary of their perspective.
First of all, you should not get too caught up in the terminology. In statistics, jargon should never be used as a substitute for a mathematical understanding of the models themselves. That is especially true for random and mixed effects models. "Mixed" just means the model has both fixed and random effects, so let's focus on the difference between fixed and random.
Mathematically, the difference between fixed and random effects modeling is that the latter uses a multilevel approach to estimate an effect's variation across multiple "groups" (i.e. observations grouped by levels of a categorical predictor).*
In practice this means that, if you have few data points in a group, the group's effect estimate will be based partially on the more abundant data from other groups. This is known as partial pooling, and it's a nice compromise between completely pooling all groups, which could mask group-level variation, and treating all groups completely separately, which could give poor estimates for low-sample groups.
An example should help prime your intuition. Suppose you want to estimate average US household income by ZIP code. You have a large dataset containing observations of households' incomes and ZIP codes. Some ZIP codes are well represented in the dataset, but others have only a couple households.
For your initial model you would most likely take the mean income in each ZIP. This will work well when you have lots of data for a ZIP, but the estimates for your poorly sampled ZIPs will suffer from high variance. You can mitigate this by using a shrinkage estimator, which will push extreme values towards the mean income across all ZIP codes.
But how much shrinkage should you do for a particular ZIP? Intuitively, it should depend on the following:
If you model ZIP code as a random effect, the mean income estimate in all ZIP codes will be subjected to a statistically well-founded shrinkage, taking into account all the factors above.
The best part is that random and mixed effects models automatically handle (4), the variability estimation, for all random effects in the model. This is harder than it seems at first glance. You might think you can estimate the variance by just taking the sample mean across all ZIPs, but this will be biased high, because low-sample ZIPs will contribute spurious variance. (Imagine sampling one person from Seattle - you might get Bill Gates!) In a random effects model, this bias does not occur because it the model and inference process understand how to account for it.
Having accounted for (1)-(4), a random/mixed effects model is able to determine the appropriate shrinkage for low-sample groups. It can also handle much more complicated models with many different predictors.
If this sounds like hierarchical Bayesian modeling to you, you're right - it is a close relative but not identical. Mixed effects models are hierarchical in that they have distributions for underlying parameters, but they are typically not fully Bayesian because the top-level hyperparameters will not be given proper priors. For example, in the above example we would most likely treat the mean income in a given ZIP as a sample from a normal distribution, with unknown mean and sigma to be estimated by the mixed-effects fitting process. However, a (non-Bayesian) mixed effects model will typically not have a prior on the unknown mean and sigma, so it's not fully Bayesian. That said, with a decent-sized data set, the standard mixed effects model and the fully Bayesian variant will often give very similar results.
*To fully appreciate the power of mixed effects and multilevel modeling, it is important to realize that the concept of group is very flexible. All it means is a set of observations that share a common value of a categorical predictor variable. A group could be composed of multiple observations of a single person, or multiple people in a school, or multiple schools in a district, or multiple varieties of a single kind of fruit, or multiple kinds of vegetable from the same harvest, or multiple harvests of the same kind of vegetable... the possibilities are endless!