# What is the mathematical difference between random- and fixed-effects?

I have found a lot on the internet regarding the interpretation of random- and fixed-effects. However I could not get a source pinning down the following:

What is the mathematical difference between random- and fixed-effects?

By that I mean the mathematical formulation of the model and the way parameters are estimated.

-
 Well, fixed effects effect the mean of a joint distribution and random effects effect the variance and association structure. What exactly do you mean by the "mathematical difference"? Are you asking how the the likelihood changes? Can you be more specific? – Macro Apr 10 '12 at 22:12

In a standard software package (e.g. R's lmer), the basic difference is:

• fixed effects are estimated by maximum likelihood (least squares for a linear model)
• random effects are estimated by empirical Bayes (least squares with some shrinkage for a linear model, where the shrinkage parameter is chosen by maximum likelihood)

If you're being Bayesian (e.g. WinBUGS), then there is no real difference.

-
 Simon, the answer is excellent. For application of Anova type models, we must postulate either a fixedeffects assumption or a random effects assumption, Both should work. – subhash c. davar Apr 26 at 14:01

@Joke A fixed-effects model implies that the effect-size generated by a study(or experiment) is fixed i.e. repeat measurements for an intervention turn out same effect-size.Presumably, the external and internal conditions for the experiment do not change. If you have a number of trials and or studies under different condtions, you will have different effect-sizes. The parametric estimates of mean and variance for a set of effect-sizes can be realised by either presuming that these are fixed-effects or these are random-effects(realised from a super-population). If you have data or could explain abit of it, I shall try to persist. What is the real theme in your mind, I think that it is matter that can be resolved with the help of mathematical statistics.

-

In econ land, such effects are individual-specific intercepts (or constants) that are unobserved, but can be estimated using panel data (repeated observation on the same units over time). The fixed effects estimation method allows for correlation between the unit-specific intercepts and the independent explanatory variables. The random effects does not. The cost of using the more flexible fixed effects is that you cannot estimate the coefficient on variables that are time-invariant (like gender, religion, or race).

N.B. Other fields have their own terminology, which can be rather confusing.

-
(-1) this says nothing about the mathematical difference between fixed and random effects – Macro Apr 10 '12 at 22:08
@Macro Agreed. Before that comes up, it would be helpful to know if the econ terminology is what the OP is looking for. I should have been clearer on that. – Dimitriy V. Masterov Apr 10 '12 at 22:15
OK. In that case this may be more appropriate as a comment, wouldn't you say? – Macro Apr 10 '12 at 22:15
The statement "The cost of using the more flexible fixed effects is that you cannot estimate the coefficient on variables that are time-invariant" just isn't true. I just did a simulation where you have repeated measurements on individuals and a single binary predictor that is not time varying. If you include a fixed effect for ID and one for the binary predictor, you most certainly can estimate the coefficient on the binary predictor (although, I will admit, if you don't have a lot of repeated measurements, the estimate does have a large standard error). – Macro Apr 10 '12 at 22:45
Andrew Gelman (who is not an economist), lists 5 distinct definitions in his ANOVA paper: stat.columbia.edu/~gelman/research/published/banova7.pdf. – Dimitriy V. Masterov Apr 10 '12 at 23:07