Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 5 down vote favorite
2

I have found a lot on the internet regarding the interpretation of random- and fixed-effects. However I could not get a source pinning down the following:

What is the mathematical difference between random- and fixed-effects?

By that I mean the mathematical formulation of the model and the way parameters are estimated.

share|improve this question
Well, fixed effects effect the mean of a joint distribution and random effects effect the variance and association structure. What exactly do you mean by the "mathematical difference"? Are you asking how the the likelihood changes? Can you be more specific? – Macro Apr 10 '12 at 22:12

3 Answers

up vote 1 down vote

In a standard software package (e.g. R's lmer), the basic difference is:

  • fixed effects are estimated by maximum likelihood (least squares for a linear model)
  • random effects are estimated by empirical Bayes (least squares with some shrinkage for a linear model, where the shrinkage parameter is chosen by maximum likelihood)

If you're being Bayesian (e.g. WinBUGS), then there is no real difference.

share|improve this answer
Simon, the answer is excellent. For application of Anova type models, we must postulate either a fixedeffects assumption or a random effects assumption, Both should work. – subhash c. davar Apr 26 at 14:01
up vote 0 down vote

@Joke A fixed-effects model implies that the effect-size generated by a study(or experiment) is fixed i.e. repeat measurements for an intervention turn out same effect-size.Presumably, the external and internal conditions for the experiment do not change. If you have a number of trials and or studies under different condtions, you will have different effect-sizes. The parametric estimates of mean and variance for a set of effect-sizes can be realised by either presuming that these are fixed-effects or these are random-effects(realised from a super-population). If you have data or could explain abit of it, I shall try to persist. What is the real theme in your mind, I think that it is matter that can be resolved with the help of mathematical statistics.

share|improve this answer
up vote -1 down vote

In econ land, such effects are individual-specific intercepts (or constants) that are unobserved, but can be estimated using panel data (repeated observation on the same units over time). The fixed effects estimation method allows for correlation between the unit-specific intercepts and the independent explanatory variables. The random effects does not. The cost of using the more flexible fixed effects is that you cannot estimate the coefficient on variables that are time-invariant (like gender, religion, or race).

N.B. Other fields have their own terminology, which can be rather confusing.

share|improve this answer
(-1) this says nothing about the mathematical difference between fixed and random effects – Macro Apr 10 '12 at 22:08
1  
@Macro Agreed. Before that comes up, it would be helpful to know if the econ terminology is what the OP is looking for. I should have been clearer on that. – Dimitriy V. Masterov Apr 10 '12 at 22:15
OK. In that case this may be more appropriate as a comment, wouldn't you say? – Macro Apr 10 '12 at 22:15
The statement "The cost of using the more flexible fixed effects is that you cannot estimate the coefficient on variables that are time-invariant" just isn't true. I just did a simulation where you have repeated measurements on individuals and a single binary predictor that is not time varying. If you include a fixed effect for ID and one for the binary predictor, you most certainly can estimate the coefficient on the binary predictor (although, I will admit, if you don't have a lot of repeated measurements, the estimate does have a large standard error). – Macro Apr 10 '12 at 22:45
3  
Andrew Gelman (who is not an economist), lists 5 distinct definitions in his ANOVA paper: stat.columbia.edu/~gelman/research/published/banova7.pdf. – Dimitriy V. Masterov Apr 10 '12 at 23:07
show 4 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.