I have some data that I would like to model in a hierarchical function (it's a toy problem). In this case its production rates on two production lines, their associated products and how much they make in a run. Some products I have lots of data for (they run weekly) and others less so (run monthly). In theory the production rates should be the same for the products of each machine however in practice some achieve greater production rates than others. You also find efficiency gains for long production runs (generally you have inefficiencies at start up). Therefore the levels would be:

1) Machine (Machine 1, Machine 2)

2) Run Length (0 - 10,000)

3) Product (Prod 1, Prod 2, Prod N)

I have been looking at multi-level regression and when I look at the formulas I start to wonder is it equivalent to performing regression on the error of each level like this:

Step 1:

Fit data to y1_pred = a1 mach1 + b1 mach2 where mach1=1 and mach2=0 when data is from machine 1 and visa versa

Step 2:

Calculate y1_pred for each sample based on the above formula (with derived a,b) and subtract it from each data point, y_act. Let's call the difference y1_diff

Step 3:

Train y2_pred = a2 mach1 runlength + b2 mach2 runlength + c2 against the derived datapoints y1_diff where mach1=1 and mach2=0 when data is from machine 1 and visa versa

Step 4:

Calculate y2_pred for each data point and subtract it from the data points y1_diff and call the result y2_diff

Step 5

Train y3_pred = a3 mach1 runrate prod1 + b3 mach2 runrate prod2 .. + against the derived data points y2_diff

Step 6: The final model is y1_pred + y2_pred + y3_pred

The reason why it interests me is that I could use different model types for each level - for example - linear regression on level 1, decision tree on level 2 and random forest on level 3. I could also iteratively tune regularization parameters at each level.

So is the above equivalent to multilevel regression?

Thanks, Simeon

  • $\begingroup$ Your proposed steps for HLM model building doesn't fit with my intuition about them. What literature about this class of models have you read that suggests such a structure? $\endgroup$ – Mike Hunter Jan 28 '18 at 13:33
  • $\begingroup$ When I look at en.m.wikipedia.org/wiki/Multilevel_model and look at the equations for level 1 and level 2 regression. They seem to be just a regression across the whole data set with the level 1 predictor and then a regression of the level 2 predictor against the difference between the predicted values using the level 1 predictor and the actual values. $\endgroup$ – simeon Jan 28 '18 at 18:34
  • $\begingroup$ I would revert to Judith Singer's paper Using SAS Proc Mixed to Fit Multilevel Models as a more canonical, introductory source for HLMs. $\endgroup$ – Mike Hunter Jan 29 '18 at 0:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.