Comparing the random effects and fixed effects models

Question

Consider the random effects model $y_{it} = x_{it}'\beta + \mu_i + \nu_{it}$ where the composite error is $\mu_i + \nu_{it}$. We transform the variables and the error term of the regression equation using $\lambda = 1 - \sqrt{\frac{\sigma^2_\nu}{\sigma^2_\nu+T\sigma^2_\mu}}$. For example the transformed dependent variable is $y_{it}-\lambda\bar{y}_i$. Books give the typical story that if $\sigma^2_{\mu}$ is very large, then $\lambda$ is going to 1 and $y_{it}-\lambda\bar{y}_i$ becomes $y_{it}-\bar{y}_i$ which is the within transformation and hence we end up with the fixed effects model. I have two questions.

Question 1. Why in the random effects model we assume that $\mu_i$ is a random variable whereas in the fixed effects model it is not random? In both models $\mu_i$ differ across individuals. Because the books give an interpretation to $\sigma^2_{\mu}$ above and make the fixed effects model as a special case of $\sigma^2_{\mu}$ being very large or being close to 0. So this suggests that we could in fact treat $\mu_i$ as random in the fixed effects model. Why do not we?

Question 2. Why the fixed effects model is called a fixed effects model? What is fixed in the model? $\mu_i$ does not vary within individuals and therefore is fixed? But this is also the case in the random effects model.

Question 3. If between $\mu_i$ is of interest in a fixed effects model, why would the fixed effects estimator exploit the within variation? So in the observables of the regression, the fixed effects estimator exploits the within variation, but when it comes to the error term the fixed effects model is concerned about the variation in $\mu_i$ across individuals. Why is this not a contradiction?

Answer 1

Maybe an intuitive way to think about fixed versus random effects would be to imagine what would happen if you were to repeat your study under similar condions a large number of times.

Fixed Effect

You are interested in how study time impacts reading scores for students enrolled in all 3 schools in your local area: School 1, School 2 and School 3. The 3 schools are the only ones you are interested in. If you were to repeat your study many times, the schools would be the same across your many (random) study samples. In other words, the levels of of the categorical factor school - School 1, School 2 and School 3 - would be fixed across study samples.

Random Effect

You are interested in how study time impacts reading scores for students enrolled in a random sample of 3 schools in your local area. The 3 schools represent a larger pool of schools you are interested in, so you are not interested in them other than the fact they tell you something about the pool of schools. If you were to repeat your study many times, the schools would be the different across your many (random) study samples. The first study sample might include School 1, School 5 and School 8. The second study sample might include School 2, School 4 snd School 10, etc. In other words, the levels of of the categorical factor school represented in any particular study sample would be a random sample from the entire pool of levels.

Putting it all together

In practice, you only have one study sample to work with, so that can make it a bit harder to understand why you treat the effects of a grouping factor such as school as fixed or random. But nothing is stopping you from asking yourself: If I repeated my study under similar conditions, would I get to see the exact same levels of the grouping factor or totally different levels? If the exact same levels, I can treat the levels of the grouping factor seen in the study sample as fixed, knowing these levels are the only ones I am interested in for my study. If totally different levels, I can view the levels of the grouping factor seen in the study sample as a random subset of the larger pool of levels I am really interested in for my study.

Addendum:

Let's say you have a response variable, Weight, you measure for each of a series of children at the same times, where the times consist of Time = 0 (Birth), Time = 1 (5 years), Time = 2 (10 years) and Time = 3 (15 years). If you plot Weight vs. Time for each child, let's say you see an increasing linear relationship with the same slope for each child. In other words, Weight tends to increase as a function of time for each child. Let's also say that Weight increases at the same rate for each child. However, each child starts with a different Weight at birth - some children might start with a weight of 4kg, others with a weight of 3.5kg, others with a weight of 4.5 kg, etc.

If you fit a linear mixed effects model to these data, that is akin to fitting a series of linear model - one for each child - but where you allow the child-specific intercepts to deviate from the intercept for the "average child" by an amount $\mu_i$. For a particular child, this amount could be negative (meaning the child starts out with a below-average birth weight), zero (meaning the child starts out with an average birth weight) or above average (meaning the child starts out with an above-average birth weight). This deviation $\mu_i$ changes randomly from child to child, so you will impose the constraint that it actually follows a normal distribution with mean 0 and a particular, unknown standard deviation $\sigma$.

By imposing this constraint, you are essentially saying that the average child has a deviation of 0 from the average weight at birth and that an equal number of children in your target population have below-average birth weight and above average-birth weight. You are also saying that the chance that the next child you'd look at has below or above average weight at birth is random. Finally, you are saying that 95% of the children in your target population have deviations $\mu_i$ falling within +/- 2 $\sigma$ from 0.

To sum up, you are treating $\mu_i$ as a random intercept effect. This random effect is meant to capture the net effect of all the (unobserved or perhaps observed but not currently included in the model) subject-specific influences which conspire to affect how large/small a baby's weight is at birth. Usually we treat $\mu_i$ as a random intercept effect when the babies included in the study are selected so as to represent a larger pool of babies. So it's not just that the $\mu_i$'s differ across individuals - they would have to differ randomly from one individual to another (i.e., in unpredictable rather than systematic ways). This model is called a random effects model because the birth weights of the babies are assumed to vary randomly from one baby to another. In practice, you would want to have at least 5 babies to contemplate such a model and the 5 babies would have to be selected at random at birth - perhaps from the same hospital and then followed up until they are 15-years old.

In a fixed effects setting, you would only focus on a small number of babies but those would be the only ones you would care about and you wouldn't want to generalize the findings of your fixed effects model to any other children from a larger pool - you would simply be interested in those 5 concrete children. If the random effects model assumes that children are somewhat similar - they all have birth weights in the vicinity of the average birth weight, the fixed effects model does not make any assumption of similarity among birth weights of concrete children. In fact, it treats those birth weights as distinct quantities which have nothing to do with each other.