Random effect estimates for factor levels that don't exist

Question

I am modeling the effect of race on test scores and would like to use a mixed or nested linear model to obtain estimates of the interaction between race and the school a student attends. I have specified a model:

lmer(TEST_SCORE ~ PRIOR_TEST_SCORE + (RACE | SCHOOL))

• Race is a factor with three levels: Black, White, Hispanic
• School is a factor with 150 levels
• Test scores are numeric and continuous

The challenge is that some schools lack diversity, and not all races are present within all schools. When run as a fixed effects model I get NA's for the coefficients as I would expect, since those races do not exist within those schools. When I run as a mixed model and look at those same schools' estimates by race using ranef, I find that estimates exist for races that don't attend that school.

Why is this happening and how is lmer estimating random effects for combinations that do not exist in the data? Is it not possible to use this model when all levels are not present in each nested group?

Answer 1

The model you are fitting postulates that per SCHOOL you have a different random intercept per RACE (BTW perhaps it would be more easier to interpret if you exclude the intercept in the random-effects part, i.e., lmer(TEST_SCORE ~ PRIOR_TEST_SCORE + (0 + RACE | SCHOOL))).

However, the formulation you have used also postulates that the random intercepts for the different races are correlated, i.e., the covariance matrix for the random effects is an unstructured $3 \times 3$ matrix.

The estimates of the random effects your get from ranef() are from the conditional/posterior distribution of the random effects given the observed data. Now, even if you don’t have data for a particular race in a particular school, you had assumed that the random effects of the three races are correlated. Hence, when you estimate the random effects for the races for which you do have data, these estimates will also tell you something for the random effect of the race for which you don’t have data.