Determining significant terms in lme using anova(lme.model1) vs anova(lme.model1, lme,model2)

Question

I am performing linear mixed effects analysis on 4 time points (same individuals), with BMI and race as fixed effects in R using lme(). I am interested in both the main effects and interactions and have included a term for within subject random effects. My model is as follows:

   lme.model1<-lme(outcome~visit+bmi+race+visit*bmi+visit*race+bmi*race+visit*bmi*race, random=~1|subject,data=lme_data,na.action=na.exclude)

I then want to determine which main effect and interaction terms are significant. I have approached this by first building the model excluding the term I am interested in, for example if I wanted to know if the race term has a significant effect I would use the following:

lme.model2<-lme(outcome~visit+bmi+visit*bmi, random=~1|subject,data=lme_data,na.action=na.exclude

anova(lme.model1,lme.model2)
            Model df AIC       BIC    logLik      Test L.Ratio p-value
lme.model1     1 26 11.166337 107.7234 20.41683                       
lme.model2     2 14  3.879536  55.8718 12.06023 1 vs 2 16.7132  0.1607

However, using:

anova(lme.model1)
               numDF denDF  F-value p-value
(Intercept)        1   205 41515.74  <.0001
visit              3   205    19.88  <.0001
bmi                2    74     4.63  0.0128
race               1    74     6.59  0.0123
visit:bmi          6   205     2.74  0.0140
visit:race         3   205     2.22  0.0869
bmi:race           2    74     0.06  0.9436
visit:bmi:race     6   205     0.42  0.8633

I know the first example comparing the two models uses a likelihood ratio and the second uses anova with type 1 SS (which from what I read is preferable when you want to look at interactions). However, since these give different results and interpretations for race as a main effect, what is the most acceptable method of determining significance of main effects (and interactions) in my case? Is there an acceptable 'norm'?

Answer 1

I am generalizing here. We usually begin to build the complex model (including interactions). Then elimit the interactions first, if they are not significant. Then start comparing models with/without fixed effects (including their interactions) of interest. Comparing on a model level gives the opportunity to test for complex differences at once, e.g. include/exclude multiple fixed effects and/or interactions (if that may be of interest to you).

Note that when you test multiple effects at once, particularly in a small sample size, the LR based testing may give unexpectedly small p-values ("anticonservative"; Pinheiro & Bates (2000) Mixed-effects Models in S and S-PLUS. p.88). Thus the F test may be better when a small sample size is given and multiple parameters are to be removed from the model.

I'm definitely interested to know if people follow different strategies.