2
$\begingroup$

I am developing a model to forecast the number of students enrolled in roughly 65 primary schools in a large city. Relevant predictors include the number of appropriately aged children living in the neighborhood of the city and whether the school offers to supervise children in the afternoon.

I use data from the past 10 years to forecast the next 5. Future information on the two predictors is available through a separately conducted demographic forecast and knowledge of future plans for the introduction of afternoon activities.

The effect of children living in the neighborhood on subsequent first graders is large across the entire sample, but it varies strongly between schools. To a lesser extent, this is also true for the afternoon supervision variable. For this reason, I have thus far estimated linear mixed effects models with random coefficients for both variables on the school level. I also include a random intercept on the year level to address contemporaneous correlation.

Thus, my basic model looks like this (using lme4 syntax):

lme(no_students ~ no_children + dum_afternoon + (no_children + dum_afternoon | school_id) + (1 | year))

This model performs reasonably well, with a MAE of ~4.5 students (the mean of the predicted variable is 71). I tested this fit against a number of alternative models, including one using differenced data and one where no_students and no_children were both log-transformed.

None of the alternative models perform better than the original model in predicting in-sample, but when I split up the data into a training and a testing set, the model with the log-transformed variables turned out to be much better in predicting out-of-sample (once of course predicted values were transformed back to the original scale). To be more precise, the MAE for the original model just about doubled while for the log model, it only went to 6-7.

I am having trouble understanding why this should be the case. Is there something inherent in log-log models that makes them better suited for out-of-sample forecasting?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.