- Can someone help me to understand fixed/random effect models? You may either explain in your own way if you have digested these concepts or direct me to the resource (book, notes, website) with specific address (page number, chapter etc) so that I can learn them without any confusion.
- Is this true: "We have fixed effects in general and random effects are specific cases"? I would especially be grateful to get help where the description goes from general models to specific ones with fixed and random effects
$\begingroup$
$\endgroup$
3
-
2$\begingroup$ of possible interest: stats.stackexchange.com/questions/4700/… $\endgroup$– ocramCommented Aug 9, 2012 at 13:11
-
$\begingroup$ See the book references on the mixed-model tag. Number 1 is addressed in (some) introductory chapter for all multi-level modelling books I have read. $\endgroup$– Andy WCommented Aug 9, 2012 at 13:40
-
1$\begingroup$ Of possible interest: What is the difference between random effects-, fixed effects- & marginal model? $\endgroup$– gung - Reinstate MonicaCommented Nov 24, 2014 at 20:27
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
2
This seems a great question as it touches a nomenclature issue in econometrics that disturbs students when switching to statistic literature (books, teachers, etc). I suggest you http://www.amazon.com/Econometric-Analysis-Cross-Section-Panel/dp/0262232197 chapter 10.
Assume that your variable of interest $y_{it}$ is observed in two dimensions (e.g. individuals and time) depends on observed characteristics $x_{it}$ and unobserved ones $u_{it}$. If $y_{it}$ are observed wages then we may argue that it's determined by observed (education) and unobserved skills (talents, etc.). But it's clear that unobserved skills may be correlated with educational levels. So that leads to the error decomposition: $u_{it} = e_{it}+v_i$ where $v_i$ is the error (random) component that we may assume to be correlated with the $x$'s. i.e. $v_i$ models the individual's unobserved skills as a random individual component.
Thus the model becomes:
$y_{it} = \sum_j\theta_jx_j + e_{it}+ v_{i} $
This model is usually labeled as a FE model, but as Wooldridge argues it would be wiser to call it a RE model with correlated error component whereas if $v_i$ is not correlated to the $x's$ it becomes a RE model. So this answer your second question, the FE setup is more general as it allows for correlation between $v_i$ and the $x's$.
Older books in econometrics tend to refer to FE to a model with individual specific constants, unfortunately this is still present in nowadays literature (I guess that in statistics they never have had this confussion. I definitevely suggest the Wooldridge lectures that develops the potential missunderstanding issue)
-
$\begingroup$ Thanks for link to (1) Excellent Resource and (2) nice explanation $\endgroup$– Stat-RCommented Aug 10, 2012 at 12:33
-
1$\begingroup$ This is a different way of explaining these ideas than I'm used to seeing, but really nicely done. +1 $\endgroup$ Commented Aug 11, 2012 at 3:53
$\begingroup$
$\endgroup$
3
My best example of a random effect in a model comes from clinical trial studies. In clinical trial we enroll patients from various hospitals (called sites). The sites are selected from a large set of potential sites. There can be site related factors that effect the response to treatment. So in a linear model you often would want include site as a main effect.
But is it appropriate to have site as a fixed effect? We generally don't do that. We can often think of the sites that we selected for the trial as a random sample from the potential sites we could have selected. This may not be quite the case but it may be a more reasonable assumption than assuming the site effect is fixed. So treating site as a random effect allows us to incorporate the variability in the site effect that is due to picking a set of k sites out of a population containing N sites.
The general idea is that the group is not fixed but was selected from a larger population and other choices for the group were possible and would have led to different results. So treating it as a random effect incorporates that type of variability into the model that you would not get from a fixed effect.
answered Aug 9, 2012 at 14:53
-
$\begingroup$ @ocram reference is quite interesting. It points out the heterogeneity regarding the FE definitions. But, to which definition is Stat-R refering to ? His second questions suggests that FE is considered as a RE with correlated random component. Under that definition and within your example, a FE would mean that a treatment could be correlated with an unobserved (or omiited) site effect, right? $\endgroup$– JDavCommented Aug 9, 2012 at 22:52
-
2$\begingroup$ Nice - your last paragraph is a very succinct way of putting it. +1 $\endgroup$– LukeCommented Mar 22, 2013 at 15:25
-
4$\begingroup$ @MichaelChernick: nice example. So you argue that hospital site should be treated as a random and not as a fixed effect. But what would be the actual difference in the outcome between these two options? If we treat it as fixed, then we will get a regression coefficient for each hospital, and can test e.g. if the main effect of hospital is significant. If we treat is a random, we will not get a regression coeff for each hospital (correct?); can we still test the main effect of hospital? More importantly, does it maybe increase/decrease power of other main effects/interactions in the model? $\endgroup$– amoebaCommented Feb 26, 2014 at 23:10
$\begingroup$
$\endgroup$
0
- Not sure about a book but here is an example. Suppose we have a sample of birth weights from a large cohort of babies over a long period of time. The weights of babies born to the same women would be more similar than the weights of babies born to different mothers. Boys are also heavier than girls.
So, a fixed effects model ignoring correlation in weights among babies born to the same mother is:
Model 1. mean birth weight = intercept + sex
Another fixed effects model adjusting for such correlation is:
Model 2. mean birth weight = intercept + sex + mother_id
However, firstly we might not be interested in the effects for each particular mother. Also, we consider the mother to be a random mother from the population all mothers. So we construct a mixed model with a fixed effect for sex and a random effect (i.e. a random intercept) for the mother:
Model 3: mean birth weight = intercept + sex + u
This u will be different for each mother, just as in Model 2 but it is not actually estimated. Rather, only its variance is estimated. This variance estimate gives us an idea as to the level of clustering of weights by mother.
Hope that makes some sense.
