Tissue samples were taken from 4 differention locations and repeatedly measured. This was done identically for 3 animals. The research question was: Are there differences in measurement between the locations?

=> Repeated-measures design, with location as within-subjects factor

m1 <- lmer(meas ~ location + (1 + location | animal))  

I would like to test for a main effect of location by a likelihood ratio test via anova(model, nullmodel) (I am aware of the differing opinions regarding this).

But I am unclear, which, or if any, of the following two is the correct null model:

null.1 <- lmer(meas ~ 1 + (1 | animal), REML=FALSE)
null.2 <- lmer(meas ~ 1 + (1 + location | animal), REML=FALSE)

I am inclinced to think it’s null.1, but I am not entirely sure.

Formulated more generally: When specifying a maximal random effects structure for a within-subjects factor, what is the null model for that factor for a maximum-likelihood ratio test?

  • Just as a reminder: the distribution of the LRT for variance components is a non-standard mixture of chi-square distributions with different degrees of freedom. – StasK Sep 8 '14 at 20:57
up vote 2 down vote accepted

For building models, West, Welsh, and Galecki (2014) propose 2 strategies: step-up and top-down. In the step-up strategy your first step is to find the best "unconditional" model: you start with including all statistically significant as well as theoretically relevant random effects but only the fixed intercept. Then test $H_0: \sigma^2_{random}=0$ using the likelihood test. If the $H_0$ cannot be rejected, remove the random effect from the model. Afterwards, you start including level 1 covariates and remove non-significant ones, and then level-2, and so on.

So in your case, you may want to find the best "unconditional" model by running

null.1_reml <- lmer(meas ~ 1 + (1 | animal), REML=TRUE)
null.2_reml <- lmer(meas ~ 1 + (1 + location | animal), REML=TRUE)

You cannot simply use anova(null.1_reml, null.2_reml) because the the test statistic is distributed as a mixture of $\chi^2_1$ and $\chi^2_2$ distributions with equal weights. So use this instead:

0.5*(1 - pchisq(x, 1)) + 0.5*(1 - pchisq(x, 2))

where x is the difference in the log-likelihood value between the 2 models.

This tests

$ H_0: D= \left( \begin{matrix} \sigma^2_{int} & 0 \\ 0 & 0 \end{matrix} \right) $

$ H_A: D= \left( \begin{matrix} \sigma^2_{int} & \sigma_{int,location} \\ \sigma_{int,location} & \sigma^2_{location} \end{matrix} \right) $

where $\sigma_{int,location}$ is the covariance between the intercept and the location effect. If you want to test them individually, you can also fit a model with only the location effect and the intercept without their covariance. After finding the best unconditional model proceed to include fixed effects.

The top-down strategy recommends you first add fixed effects of as many covariates of interest as possible and remove non-significant ones to find the best mean model. Then include random effects.

Reference

West, B.T., Welch, K.B. and Galecki, A.T. (with Contributions from Brenda W. Gillespie) (2014). Linear Mixed Models: A Practical Guide using Statistical Software, Second Edition. Chapman Hall / CRC Press: Boca Raton, FL.

  • Thank you for this Masato, it’s quite instructional (and reminds me to read West et al. thoroughly). Do I understand you correctly, then, in stating that after having arrived at a final model, the correct way to test for a main effect of a (categorical) predictor, such as location, is: “To perform [a likelihood ratio test], one compares a model containing the fixed effect of interest to a model that is identical in all respects except the fixed effect in question. One should not also remove any random effects associated with the fixed effect when making the comparison.” (Barr et al., 2013) – coanil Sep 8 '14 at 19:25
  • (I have just stumbled upon this) And therefore, assuming best.fit <- lmer(meas ~ location + (1 + location | animal) is indeed “the best” model, the appropriate null model is null.2 <- lmer(meas ~ 1 + (1 + location | animal)? I am aware that this test for significance itself is rendered redundant if this best model was indeed arrived at by the the model-building strategy you recommend, but sometimes people demand to see p-values.. – coanil Sep 8 '14 at 19:26
  • 1
    Then you may want to use the top-down strategy. That is, compare null.1 <- lmer(meas ~ 1 + (1 | animal), REML=FALSE) to full.1 <- lmer(meas ~ location + (1 | animal), REML=FALSE) to show the location fixed effect is significant, then compare full.1 to null.2 after refitting full.1 with REML. Then you can avoid testing the location random effect twice. – Masato Nakazawa Sep 8 '14 at 19:52

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.