Assume I have two factors A and B potentially predicting my outcome Y. Now I would like to test for fixed-effects using likelihood ratio test to find the best model.

fm1 <- lmer(Y~1+A*B+(1|subject))
fm2 <- lmer(Y~1+A+B+(1|subject))
fm3 <- lmer(Y~1+A+(1|subject))
fm4 <- lmer(Y~1+B+(1|subject))
fm5 <- lmer(Y~1+(1|subject))

anova(fm1, fm2, fm3, fm4, fm5)

However I would also like to test for different random-effect specifications.

# all possible random-effects specifications for fm1
fm6 <- lmer(Y~1+A*B+(1+A*B|subject)) 
fm7 <- lmer(Y~1+A*B+(1+A|subject)+(1+B|subject))
fm8 <- lmer(Y~1+A*B+(1+A|subject))
fm8 <- lmer(Y~1+A*B+(1+B|subject)) 
fm8 <- lmer(Y~1+A*B+(1|subject)) 

Is it a valid approach to first detect which fixed-effects specification is most predictive [anova(fm1, fm2, fm3, fm4, fm5)] and then to try different random-effects specification with this model? Or would only the most comprehensive approach be valid in which I specify all possible combinations of fixed- and random-effects to compare all these model to find the best fit?

  • $\begingroup$ I once had a communication with Mike Lawrence on the ez4r mailing list in which he cited Bates & Pinheiro's recommendation to not use likelihood ratio tests for models differing only in their fixed effects: groups.google.com/forum/#!msg/ez4r/GQTEtNziRwE/upOoW_TLExcJ So perhaps your first idea is already bogus, but I am no expert on this. $\endgroup$ – Henrik Apr 23 '12 at 9:36
  • $\begingroup$ Zuur is working in {nlme} and not {lme4}. It would be really useful to have some input regarding this in the lme4 package! $\endgroup$ – Dalal_EL_Hanna Nov 27 '13 at 15:09
  • $\begingroup$ Welcome to the site, @Dalal Hanna. This is not an answer to the OP's question. Please only use the "Your Answer" field to provide answers. If you have your own question click the ASK QUESTION at the top & ask it there; then we can help you properly. Since you are new here, you may want to read our about page, which has information for new users. $\endgroup$ – gung - Reinstate Monica Nov 27 '13 at 15:10

As far as I'm aware, you want to specify the random effects structure first. Chapter 5 section 7 of Zuur details the general steps for selecting a model. First, include all the relevant fixed effects. Second, identify the random effects structure using REML. Third, find the fixed effects structure with ML. Fourth, Zuur suggests using the REML estimation to present (I'm not sure why). Section 5.8 has examples.

  • $\begingroup$ @jokel This looks like a good answer, but it's usually best to wait a day or two before marking a best answer so that users from around the world get a chance to answer. Not a strict rule by any means, but just a general suggestion for this site. $\endgroup$ – Matt Parker Apr 23 '12 at 15:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.