This is pretty general, but what are the pros and cons of including additional levels in multilevel model (linear mixed model)?

I have a data containing information on multilevel administrative division of the country and most of the levels are more or less of interest for me. Sample size is not a problem in here. On one hand, simpler models are in most cases better, on another, including additional levels would enable me to compare the variances on different levels. I found examples of 4-level models in the literature, but I haven't seen any practical advise on that. Could you provide any arguments and/or literature on that?

  • $\begingroup$ Some general advice: andrewgelman.com/2007/08/16/no_you_dont_nee andrewgelman.com/2012/04/17/… Try to use all the levels that you have ? Do it work? If not, why? $\endgroup$ – kjetil b halvorsen Dec 4 '14 at 12:55
  • $\begingroup$ Temporary I killed the computation after going for over an hour and consuming >40GM RAM and counting so I don't know. $\endgroup$ – Tim Dec 4 '14 at 12:57
  • 1
    $\begingroup$ Then, start with some simpler models, see what you can get from them, and build up gradually, maybe one level at a time. And leave the computer overnight! $\endgroup$ – kjetil b halvorsen Dec 4 '14 at 13:01
  • $\begingroup$ @Tim, Have you considered "compressed sensing"? If you have bottomless samples, why not uniformly random pull a sample of them and run your model fitting against the sample? If you did it a few times then you could get an idea of stability of various complexity of model vs. sample size and likely get a decent estimator for your overall system. Once you know the form and have a good start point it is much easier to adjust the fit. $\endgroup$ – EngrStudent Jan 12 '15 at 17:47
  • $\begingroup$ @EngrStudent thanks but that is not the point. With a certain sample size it is possible to estimate a model with some number of levels - it is pretty clear because after reaching some point of complexity the model is just unestimable. The question is rather on if there is a point where the estimates are untrustworthy even if "something" got estimated. In most cases you use this kind of models with a finite number of cases and a finite number of possible levels. $\endgroup$ – Tim Jan 12 '15 at 17:57

This is hard to answer without much context. But in general, parameters of additional levels will be harder to estimate. For each additional level you will need much more data, specially for the variance-covariance parameters of the higher levels. See here for a related discussion.

  • $\begingroup$ But if number of observations is not a problem, then every estimable model is fine..? $\endgroup$ – Tim Dec 4 '14 at 21:09
  • $\begingroup$ @Tim, As far as I understand, the answer is yes. However, there will be computational costs that may be prohibitive for practical purposes. And it will be harder to interpret and check the fit of more complex models as well. $\endgroup$ – Manoel Galdino Jan 13 '15 at 14:13
  • $\begingroup$ What would you have to look for when checking the fit in this case? Do you know any "warning sighs" that could be helpful in the case of multiple-level models? Thanks! $\endgroup$ – Tim Jan 13 '15 at 14:16
  • 1
    $\begingroup$ I've never seen more than 3 levels. I've only worked with 2 levels and I'm extrapolating from what I know with two levels or three levels. I'd be concerned with convergence diagnostics and also if the variance parameters are too big. It may be hard to put a good prior on these high levels variance parameters. Run some prior sensitivity checks. $\endgroup$ – Manoel Galdino Jan 13 '15 at 14:25
  • 1
    $\begingroup$ Thanks again! This topic certainly deserves a Monte Carlo study. $\endgroup$ – Tim Jan 13 '15 at 14:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.