I built a simulation of student scores on a pre-test, posttest and gave it some random effects. The simulated data is built as follows:

$postscore \sim intercept + prescore + school + transformed\_instruction + \epsilon$

where there is an assumption that there are three schools (good, medium, and bad) and they have intercepts and random effects associated wtih each as follows:

$intercept\_bad \sim \mathcal{SN}(15, 2, -15)$

$intercept\_medium \sim \mathcal{SN}(20, 4, -15)$

$intercept\_good \sim \mathcal{SN}(30, 6, -15)$

$prescore\_bad \sim \mathcal{SN}(10, 5, -15)$

$prescore\_medium \sim \mathcal{SN}(20, 10, 15)$

$prescore\_good \sim \mathcal{SN}(30, 15, 15)$

$transformed\_instruction \sim Bi(0,1)$

$\epsilon \sim \mathcal{N}(0, 5)$

I built an OLS model, which should fail at producing the correct model because it can only see the means but not the variances of the random effects. I then built a gradient boosted model (xgboost) which I think should be able to understand the variance of the random effects. However it produces a similar solution to the OLS model (see figure). I did a grid search to see if it was a hyper parameter issue and that is not the case.

I guess the question is why does the gradient boosted model not see the variance in the random effects?

enter image description here

  • 2
    $\begingroup$ did you try mixed effects models as a benchmark? if ME doesn't get a correct model, then something's wrong in your simulation setup $\endgroup$ – Aksakal Feb 14 at 15:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.