Why would xgboost not learn the variance of this simulation

I built a simulation of student scores on a pre-test, posttest and gave it some random effects. The simulated data is built as follows:

$$postscore \sim intercept + prescore + school + transformed\_instruction + \epsilon$$

where there is an assumption that there are three schools (good, medium, and bad) and they have intercepts and random effects associated wtih each as follows:

$$intercept\_bad \sim \mathcal{SN}(15, 2, -15)$$

$$intercept\_medium \sim \mathcal{SN}(20, 4, -15)$$

$$intercept\_good \sim \mathcal{SN}(30, 6, -15)$$

$$prescore\_bad \sim \mathcal{SN}(10, 5, -15)$$

$$prescore\_medium \sim \mathcal{SN}(20, 10, 15)$$

$$prescore\_good \sim \mathcal{SN}(30, 15, 15)$$

$$transformed\_instruction \sim Bi(0,1)$$

$$\epsilon \sim \mathcal{N}(0, 5)$$

I built an OLS model, which should fail at producing the correct model because it can only see the means but not the variances of the random effects. I then built a gradient boosted model (xgboost) which I think should be able to understand the variance of the random effects. However it produces a similar solution to the OLS model (see figure). I did a grid search to see if it was a hyper parameter issue and that is not the case.

I guess the question is why does the gradient boosted model not see the variance in the random effects?

• did you try mixed effects models as a benchmark? if ME doesn't get a correct model, then something's wrong in your simulation setup – Aksakal Feb 14 at 15:51