11
$\begingroup$

I have found a web page telling that for lmer:

If your random effects are nested, or you have only one random effect, and if your data are balanced (i.e., similar sample sizes in each factor group) set REML to FALSE, because you can use maximum likelihood. If your random effects are crossed, don't set the REML argument because it defaults to TRUE anyway.

I have 2 random effects in my lmer model. One is nested:

(1|Random1A/Random1B) + (1|Random2)

Should I set REML to TRUE (by default) or FALSE?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ exactly, this is related to my former question. so in my model (in which I added all variables and trying to do a stepwise backwards drop, to find final model) I should use Reml = False? Or did I get you wrong? $\endgroup$
    – borgs
    Commented Apr 1, 2017 at 0:34
  • 2
    $\begingroup$ right. If you are holding your RE constant , and just comparing fixed effect, then you should fit using ML (REML=FALSE). Once you have your final model, refit using REML. $\endgroup$
    – user20650
    Commented Apr 1, 2017 at 0:36
  • $\begingroup$ let me translate your reply for dummies (correct me if I am wrong): I should use Reml=False during the process of getting final variables. And when I decide final variables (for example 3 out of 11 variables), this time I will turn to REML=true to get intercept, estimates of variables and q values of the final model (with 3 variables i,e)? $\endgroup$
    – borgs
    Commented Apr 1, 2017 at 0:40

1 Answer 1

Reset to default
17
$\begingroup$

In my (not entirely uninformed) opinion you're getting some questionable advice, from the web page and from the comments you received.

  • you can use REML (or ML) whenever you want (regardless of the random effects structure - single vs. multiple, balanced vs. unbalanced, crossed vs. nested)
  • in simple cases (balanced/nested/etc.) REML can be proven to provide unbiased estimates of variance components (but not unbiased estimates of e.g. standard deviation or log standard deviation)
  • you cannot compare models that differ in fixed effects if they are fitted by REML rather than ML; this is why the commenter recommends that you use REML=FALSE if you're trying to do model selection
  • however, I wouldn't recommend you do model selection in the first place, certainly not if you're going to rely on the conditional confidence intervals and p-values (i.e., analyzing the refitted 'minimal adequate' model without accounting for the effects of model selection)

From my chapter in Fox et al 2015:

It’s generally good to use REML, if it is available, when you are interested in the magnitude of the random effects variances, but never when you are comparing models with different fixed effects via hypothesis tests or information-theoretic criteria such as AIC.

Improve this answer
$\endgroup$
4
  • $\begingroup$ the way an ecologist/biologist thinks & a statistican is quite different. we'r (non-statisticians) generally more interested in results rather than the way to the results. furthermore, replication of mistakes/wrong method selections in manuscripts lead to spread of these false methodologies. although, i'm willing to understand what is stated in statistics books and statistics gurus comments... sometimes I really feel that I don't know the language spoken... I found a few e-books (including B. Bolker's Ecological Models &Data in R) but any other examples/how to docs are pretty much appreciated. $\endgroup$
    – borgs
    Commented Apr 9, 2017 at 17:42
  • $\begingroup$ It looks like this a reply to a comment, but I can't see it ... I find it a bit hard to find books that are both rigorous enough to make me happy and not too technical for ecologists; bbolker.github.io/mixedmodels-misc/ecostats_chap.html has some worked examples from the book I referenced above ... $\endgroup$
    – Ben Bolker
    Commented Apr 9, 2017 at 19:26
  • $\begingroup$ @BenBolker so what if one just wants to create a model for prediction (i.e., he/she is only interested in the prediction accuracy and not magnitudes of variances). Which approach should be used : REML or ML? Does it matter? Will one affect the predictive ability at all vs the other? $\endgroup$
    – theforestecologist
    Commented Jun 28, 2017 at 21:16
  • $\begingroup$ I doubt REML vs ML has much effect on predictive accuracy (but I don't have any evidence either way). A quick, crude literature search suggests that most of the attention to this has been in the context of accuracy of genomic estimation (i.e. BLUPs/conditional modes/breeding values) ... $\endgroup$
    – Ben Bolker
    Commented Jun 28, 2017 at 22:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.