2
$\begingroup$

I am trying to understand why random effects are useful (in Linear Mixed Models). Specifically, why are they necessary and why can't we just used fixed effect dummy variables instead?

For example, if you are trying to regress the number of days that a student will be absent in a school year on those students' math test scores, and you have data clusters, such as the classroom, school, school district, etc., why would one use a random effect instead of a fixed effect indicator variable? If one added a fixed effect dummy variable for each of the n-1 classrooms and/or a fixed effect dummy variable for each of the m-1 schools, the other fixed effect, math test score, would update to reflect the effect of math test scores, classroom and school aside. This case would also be an example of a non-longitudinal dataset.

Is it because we can specify the variance/covariance structure in random effects? This seems plausible to me, as a dummy variable fixed effect would not allow one to account for the variance/covariance structure of the dummy variable. However, if one is concerned about correlated residuals, one could still use Generalized Least Squares (without any random effects) instead of a Linear Mixed Model (with random effects) to account for this.

$\endgroup$
2
$\begingroup$

Let’s say you have students clustered within classrooms (forget about the other clusters for now), and you are trying to predict math test scores, as a function of student absences. It’s true that treating “classroom” as a fixed effect (that is including n-1 dummy variables - one for each classroom) will totally control for any bias stemming from classroom level factors, as well as address any violations of OLS assumptions (like non-independence of error terms) that are due to students being clustered in classroom. So why might you want to treat classroom as a random effect instead? Here are a just few of the many possible reasons - which may or may not apply in this particular case:

Reason #1: You can control for classroom level factors individually. Precisely because treating class as a fixed effect controls for ALL class level factors (e.g. teacher age, teacher experience, size of the class, % of students who are nonwhite, etc), it means you can’t also include any of these classroom level variables in your model. But maybe you care about them! Maybe you want to create an interaction between absences and some class level variable. Treating classroom as a random effect addresses many of the problems with OLS assumptions caused by clustering but still allows you to control for variables at the clustering level.

Reason #2: A well specified random effects model is more efficient than a fixed effects model. Obviously a model with random effects has the potential to be biased due to omitted variable bias at the classroom level. But if you DID control for all important class level confounders, so that the coefficient estimates you get from a model with random effects are the same as what you get from your fixed effects model, the standard errors on the random effects model will be (correctly) smaller. So you should prefer the random effects model. In these situations we can use a Hausman test to help us decide whether to use fixed or random effects.

Reason #3: You want to see how much “class level stuff” matters at all. By running a model that ONLY includes random effects for class but no predictors you can see (in the ICC) the percent of total variation in test scores that is due to class level vs student level factors. If the ICC is really low (say below 2%) then you might be able to just forget about class altogether and run an OLS. If it’s really high then maybe you want to rethink your research question, and look more at what CLASS level factors are driving test scores. Then you can see how the ICC changes as you add different variables to see what level they are explaining variance at. Sometimes variables that SEEM to be at the student level (like student SES) might actually be explaining more variance at the class level.

Reason #4: You want to get “good” estimates of the average test score in each class, but for some classes you only have data on a few students. You can use a method called Empirical Bayes estimation to get good estimates of all classes in this situation, but it requires you to use a random effects model to do so. This is a major motivation of random effects models in (e.g.) public opinion research.

Reason #5: You want to know if the effect of absences on test scores works DIFFERENTLY in different classes. Basically you want to know if the slope/coefficient of absences varies significantly at the class level. This requires a model with two random effects - one for the constant term and one for the coefficient in question - both of which vary normally at the class level.

$\endgroup$
3
  • $\begingroup$ +1. I'd note that low ICC don't necessarily mean we can drop random effects. Low ICC + large classroom sizes and standard errors will be different if class membership is unaccounted for. $\endgroup$ – Heteroskedastic Jim Mar 17 at 12:45
  • $\begingroup$ Good point. Edited to note that you MIGHT be able to forget about clustering if the ICC is low. Devil is in the details of course. $\endgroup$ – Graham Wright Mar 17 at 13:09
  • 1
    $\begingroup$ +1, some thoughts: re #2, including all the fixed effects won't necessarily yield the same estimates as RE; re #5, you can test interactions b/t class & absences w/ fixed effects; potential #6, FE estimates are specific to those classrooms & generalizing is likely a bit sketchy; potential #7, RE gives estimates of population means, SDs, & correlations, which may be of substantive interest; potential #8, w/ multiple levels using FE gets very complicated & more difficult to extract the estimates at each level; potential #9, if you have only n=1 at individual levels, FE can be unidentifiable. $\endgroup$ – gung - Reinstate Monica Mar 17 at 14:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.