3
$\begingroup$

I have read that a good approach to mixed modeling is to try and fit the maximally complex random effects structure, and then simplify it until it converges.

I would like to be principled about this and have a procedure for simplifying the model. Is this sensible?

If it does not converge will:

1) Increase the number of iterations to 1,000,000.

If this does not work, or the fit is singular, I will:

2) Remove correlation among random effects parameters, beginning with item effects.

3) Remove random effects parameters with variance 0, and/or highest-order effects with lowest variance.

I'm wondering if it makes sense to remove the correlation between terms before trying to simplify the random effects structure. Or if the latter should come first?

$\endgroup$
2
  • 1
    $\begingroup$ This sounds more Procrustean than principled: some problems may need those parameters while others might not. Shouldn't any principled approach account for the context and objectives of the modeling? $\endgroup$
    – whuber
    Commented Nov 7, 2018 at 15:01
  • $\begingroup$ I am only every talking about removing random intercepts. In terms of slopes, there are some that I think would matter, but others that I only think might matter. Wouldn't an a priori approach like this be better than using some sort of "gut feeling" about which ones should matter? $\endgroup$
    – Dave
    Commented Nov 7, 2018 at 18:50

1 Answer 1

5
$\begingroup$

The approach of fitting mixed models with maximally complex random effects structure gained a lot of traction following the article by Barr and colleagues in 2013: "Random effects structure for confirmatory hypothesis testing: Keep it maximal", which argued, through a simulation, that such an approach will help to avoid anti-conservative conclusions.

This has since received strong criticism, notably from Douglas Bates, one of the world's foremost experts on mixed models (being the author of many papers, a book and several R & Juia packages). Instead, Bates and collaborators suggest the following approach:

We proposed (1) to use PCA to determine the dimensionality of the variance-covariance matrix of the random-effect structure, (2) to initially constrain correlation parameters to zero, especially when an initial attempt to fit a maximal model does not converge, and (3) to drop non-significant variance components and their associated correlation parameters from the model

In the same paper, they also note:

Importantly, failure to converge is not due to defects of the estimation algorithm, but is a straightforward consequence of attempting to fit a model that is too complex to be properly supported by the data.

And:

maximal models are not necessary to protect against anti-conservative conclusions. This protection is fully provided by comprehensive models that are guided by realistic expectations about the complexity that the data can support. In statistics, as elsewhere in science, parsimony is a virtue, not a vice. Bates et al (2015)

Shravan Vasishth, writing on R-Bloggers, also argues against this approach and instead favours running high power studies

References: Barr, D.J., Levy, R., Scheepers, C. and Tily, H.J., 2013. Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of memory and language, 68(3), pp.255-278.

Bates, D., Kliegl, R., Vasishth, S. and Baayen, H., 2015. Parsimonious mixed models. arXiv preprint arXiv:1506.04967.

$\endgroup$
7
  • $\begingroup$ Thanks very much for the reply! I've found Bates et al. (2015) to be extremely helpful, but I've run into an issue. When fitting a random slope for a factor with more than two levels, a slope is fitted for each contrast. I've run into an issue in which one such slope warrants inclusion while another does not. Is it possible/advisable to remove a slope for one contrast of a factor (e.g., Level B vs. Level A), but leave in another (e.g., Level C vs. Level B)? $\endgroup$
    – Dave
    Commented Nov 9, 2018 at 19:55
  • $\begingroup$ @Dave You are very welcome. That is an interesting question, so please post it as a new question. It is not in the spirit of this site to ask substantive questions such as that in comments to other questions/answers. $\endgroup$ Commented Nov 9, 2018 at 20:05
  • $\begingroup$ I actually have it posted over on Stack Overflow: stackoverflow.com/questions/53198731/… Would very much appreciate if you'd take a look. And will do, thanks again! $\endgroup$
    – Dave
    Commented Nov 9, 2018 at 20:32
  • $\begingroup$ @Dave I think that question should be re-framed in terms of a general question about mixed models and posted here, not on SO, because the premise of the question itself, is questionable $\endgroup$ Commented Nov 9, 2018 at 20:41
  • $\begingroup$ Thanks for the advice, I've done so here: stats.stackexchange.com/questions/376250/… $\endgroup$
    – Dave
    Commented Nov 9, 2018 at 21:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.