3
$\begingroup$

I am working on a mixed effects regression model where Yi = exam score of student i.

The explanatory variables are the following:

  • Level 3: school type (public vs. private) and school's socioeconomic level (numeric variable)

  • Level 2: educational model in each class (main path, education for special needs)

  • Level 1: immigration background (whether the student is an inmigrant or not), socioeconomic level of the student (numeric variable), language spoken at home (English or Other Language), and idoneity of the student (whether the student has to repeat a year or not) I use two models to explain Y. The first model does not include the variables student_socioeconomic and student_idoneity.

    model1 = lmer(data = scores, English_score ~ (1| school_id/group_id) + school_type + school_socioeconomic + group_educational_model + student_inmigrant + student_language)

    model2 = lmer(data = scores, English_score ~ (1| school_id/group_id) + school_type + school_socioeconomic + group_educational_model + student_socioeconomic + student_inmigrant + student_language + student_idoneity)

In the first model, the estimated coefficient for the variable "student_inmigrant" is positive and significant at the 1% alpha level. Yet, as I add the variables "student_socioeconomic" and "student_idoneity", the estimated coefficient for the variable "student_inmigrant" becomes negative and significant at the 1% alpha level. I believe that there is a problem of confounding variables here, but I don't know how to solve it. Could you please give me any suggestions on how to deal with this?

I have checked the VIF values in case there is multicollinearity, but the adjusted GVIF for student_inmigrant, student_idoneity and student_socioeconomic are all below 2.

Improve this question
$\endgroup$
1
  • $\begingroup$ Interesting discussion. You've disclosed results with regard to coefficients' signs and significance levels. You haven't disclosed the magnitude of effects. If your sample is very large and the coefficients, with or without the extra controls, are trivially far from zero, then this discussion becomes rather academic. $\endgroup$
    – rolando2
    Commented Jul 12 at 18:05

1 Answer 1

Reset to default
3
$\begingroup$

This does sound like a potential problem of confounding. For example, it seems quite plausible that higher income immigrants will have higher scores than low income immigrants because of differential access to resources (prior education, tutoring, etc). When you enter the additional variables, "student_socioeconomic" and "student_idoneity", into the model, you are removing from immigrant status the portion of its variance that it shares with these new variables and that overlap (correlates) with the outcome. Thus you get a more "unique" estimate of the association between immigrant status and the outcome.

The other thing I would encourage you to consider that is unique to multilevel data is that all of your lowest-level predictors, such as immigration status, themselves vary at higher levels of the data hierarchy. That is, the proportion of immigrant students could vary across classrooms and across schools. Accordingly, you can construct classroom and school means for each of the level 1 predictors and enter them into the model as level 2 and level 3 predictors. What this has the effect of doing is isolating the level-specific associations (student, class, and school). See the following threads for more information:

Difference between linear model and linear mixed model

Are covariates in a mixed model estimated between-group, within-group, or somewhere in-between? And why?

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ Hi Eric! Thank you very much for your answer :) That's indeed what I thought in the first place. Still, if that were the case, wouldn't there be a multicollinearity problem that would show in the VIF values? I also tried your second suggestion, but the individual level estimator for "inmigrant" is still positive and statistically significant, whereas the effect at the school and classroom level are not statistically significant. $\endgroup$
    – Elena García
    Commented Jul 5 at 11:29
  • $\begingroup$ Multicollinearity is an extreme situation that is quite rare in practice. If your standard errors increased exponentially when you entered the two additional predictors into the model, that would be evidence that you have a multicollinearity problem. It would have likely led to much larger confidence intervals for the immigrant coefficient. But you don't mention this, so I am inclined to believe the model is giving you a valid estimate given your data. $\endgroup$
    – Erik Ruzek
    Commented Jul 5 at 17:25
  • $\begingroup$ Thank you very much for your help, Erik! $\endgroup$
    – Elena García
    Commented Jul 12 at 7:03
  • 2
    $\begingroup$ Great answer! Lots of evidence that income and immigration are associated with private tutoring: edworkingpapers.com/sites/default/files/ai21-367.pdf. One possible interpretation of the grade repeating variable is that being forced to repeat a grade makes the student older, which improves the ability to learn and boosts the score. Or it could just be repeating the material. Immigrant students are often held back, either to give them a leg up or because they test poorly during the intake assessment. $\endgroup$
    – dimitriy
    Commented Jul 12 at 13:29
  • 2
    $\begingroup$ Hence, the immigration variable on its own conflates the adverse effects of immigration and the beneficial aging/relearning effects. Adding the controls separates the two channels. $\endgroup$
    – dimitriy
    Commented Jul 12 at 13:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.