5
$\begingroup$

I have found a web page telling that for lmer:

If your random effects are nested, or you have only one random effect, and if your data are balanced (i.e., similar sample sizes in each factor group) set REML to FALSE, because you can use maximum likelihood. If your random effects are crossed, don't set the REML argument because it defaults to TRUE anyway.

I have 2 random effects in my lmer model. One is nested:

(1|Random1A/Random1B) + (1|Random2)

Should I set REML to TRUE (by default) or FALSE?

$\endgroup$

migrated from stackoverflow.com Apr 8 '17 at 14:40

This question came from our site for professional and enthusiast programmers.

  • $\begingroup$ exactly, this is related to my former question. so in my model (in which I added all variables and trying to do a stepwise backwards drop, to find final model) I should use Reml = False? Or did I get you wrong? $\endgroup$ – borgs Apr 1 '17 at 0:34
  • 1
    $\begingroup$ right. If you are holding your RE constant , and just comparing fixed effect, then you should fit using ML (REML=FALSE). Once you have your final model, refit using REML. $\endgroup$ – user20650 Apr 1 '17 at 0:36
  • $\begingroup$ let me translate your reply for dummies (correct me if I am wrong): I should use Reml=False during the process of getting final variables. And when I decide final variables (for example 3 out of 11 variables), this time I will turn to REML=true to get intercept, estimates of variables and q values of the final model (with 3 variables i,e)? $\endgroup$ – borgs Apr 1 '17 at 0:40
7
$\begingroup$

In my (not entirely uninformed) opinion you're getting some questionable advice, from the web page and from the comments you received.

  • you can use REML (or ML) whenever you want (regardless of the random effects structure - single vs. multiple, balanced vs. unbalanced, crossed vs. nested)
  • in simple cases (balanced/nested/etc.) REML can be proven to provide unbiased estimates of variance components (but not unbiased estimates of e.g. standard deviation or log standard deviation)
  • you cannot compare models that differ in fixed effects if they are fitted by REML rather than ML; this is why the commenter recommends that you use REML=FALSE if you're trying to do model selection
  • however, I wouldn't recommend you do model selection in the first place, certainly not if you're going to rely on the conditional confidence intervals and p-values (i.e., analyzing the refitted 'minimal adequate' model without accounting for the effects of model selection)

From my chapter in Fox et al 2015:

It’s generally good to use REML, if it is available, when you are interested in the magnitude of the random effects variances, but never when you are comparing models with different fixed effects via hypothesis tests or information-theoretic criteria such as AIC.

$\endgroup$
  • $\begingroup$ the way an ecologist/biologist thinks & a statistican is quite different. we'r (non-statisticians) generally more interested in results rather than the way to the results. furthermore, replication of mistakes/wrong method selections in manuscripts lead to spread of these false methodologies. although, i'm willing to understand what is stated in statistics books and statistics gurus comments... sometimes I really feel that I don't know the language spoken... I found a few e-books (including B. Bolker's Ecological Models &Data in R) but any other examples/how to docs are pretty much appreciated. $\endgroup$ – borgs Apr 9 '17 at 17:42
  • $\begingroup$ It looks like this a reply to a comment, but I can't see it ... I find it a bit hard to find books that are both rigorous enough to make me happy and not too technical for ecologists; bbolker.github.io/mixedmodels-misc/ecostats_chap.html has some worked examples from the book I referenced above ... $\endgroup$ – Ben Bolker Apr 9 '17 at 19:26
  • $\begingroup$ @BenBolker so what if one just wants to create a model for prediction (i.e., he/she is only interested in the prediction accuracy and not magnitudes of variances). Which approach should be used : REML or ML? Does it matter? Will one affect the predictive ability at all vs the other? $\endgroup$ – theforestecologist Jun 28 '17 at 21:16
  • $\begingroup$ I doubt REML vs ML has much effect on predictive accuracy (but I don't have any evidence either way). A quick, crude literature search suggests that most of the attention to this has been in the context of accuracy of genomic estimation (i.e. BLUPs/conditional modes/breeding values) ... $\endgroup$ – Ben Bolker Jun 28 '17 at 22:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.