10
$\begingroup$

I have a GLMM of the form:

lmer(present? ~ factor1 + factor2 + continuous + factor1*continuous + 
                (1 | factor3), family=binomial)

When I use drop1(model, test="Chi"), I get different results than if I use Anova(model, type="III") from the car package or summary(model). These latter two give the same answers.

Using a bunch of fabricated data, I have found that these two methods normally do not differ. They give the same answer for balanced linear models, unbalanced linear models (where unequal n in different groups), and for balanced generalised linear models, but not for balanced generalised linear mixed models. So it appears that only in cases where random factors are included does this discord manifest.

  • Why is there a discrepancy between these two methods?
  • When using GLMM should Anova() or drop1() be used?
  • The difference between these two is rather slight, at least for my data. Does it even matter which is used?
$\endgroup$
7
$\begingroup$

I think it is the difference of which tests are computed. car::Anova uses Wald tests, whereas drop1 refits the model dropping single terms. John Fox once wrote me, that Wald tests and tests from refitted models using likelihood ratio tests (i.e., the strategy from drop1) agree for linear but not necessarily non-linear models. Unfortunately this mail was offlist and did not contain any reference. But I know that his book has a chapter on Wald tests, which could contain the desired info.

The help to car::Anova says:

Type-II tests are calculated according to the principle of marginality, testing each term after all others, except ignoring the term's higher-order relatives; so-called type-III tests violate marginality, testing each term in the model after all of the others. This definition of Type-II tests corresponds to the tests produced by SAS for analysis-of-variance models, where all of the predictors are factors, but not more generally (i.e., when there are quantitative predictors). Be very careful in formulating the model for type-III tests, or the hypotheses tested will not make sense.

Unfortunately I cannot answer you second or third question as I also would like to know that.


Update reagrding comment:

There are no Wald, LR and F tests for generalized mixed models. Anova just allows for "chisq" and "F" tests for mixed models (i.e. "mer" objects as returned by lmer). The usage section says:

## S3 method for class 'mer'
Anova(mod, type=c("II","III", 2, 3), 
    test.statistic=c("chisq", "F"), vcov.=vcov(mod), singular.ok, ...)

But as the F-tests for mer objects are calculated by pbkrtest, which for my knowledge only works for linear mixed models, Anova for GLMMs should always return chisq (hence you see no difference).

Update regarding the question:

My previous answer just tried to respond to your main question, the difference between Anova() and drop1(). But now I understand that you want to test if certainf fixed effects are significant or not. The R-sig-mixed modeling FAQ says the following regarding this:

Tests of single parameters

From worst to best:

  • Wald Z-tests
  • For balanced, nested LMMs where df can be computed: Wald t-tests
  • Likelihood ratio test, either by setting up the model so that the parameter can be isolated/dropped (via anova or drop1), or via computing likelihood profiles
  • MCMC or parametric bootstrap confidence intervals

Tests of effects (i.e. testing that several parameters are simultaneously zero)

From worst to best:

  • Wald chi-square tests (e.g. car::Anova)
  • Likelihood ratio test (via anova or drop1)
  • For balanced, nested LMMs where df can be computed: conditional F-tests
  • For LMMs: conditional F-tests with df correction (e.g. Kenward-Roger in pbkrtest package)
  • MCMC or parametric, or nonparametric, bootstrap comparisons (nonparametric bootstrapping must be implemented carefully to account for grouping factors)

(emphasis added)

This indicates that your approach of using car::Anova() for GLMMs is generally not recommended, but an approach using MCMC or bootstrap should be used. I don't know if pvals.fnc from the languageR package woks with GLMMs, but it is worth a try.

$\endgroup$
  • 1
    $\begingroup$ Thanks, Henrik. Anova( ) can compute three different tests: Wald, LR, and F. I've tried all three, but it makes no difference, which I find odd. I have a feeling it the function will refuse to use tests that it decides aren't appropriate for the data... $\endgroup$ – tim.farkas Sep 22 '12 at 17:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.