9
$\begingroup$

I want to compare two ​GLMs with binomial dependent variables. The results are: 

 m1 <- glm(symptoms ~ 1,         data=data2)
 m2 <- glm(symptoms ~ phq_index, data=data2)

The model test gives the following results: 

​ anova(m1, m2)​
         no AIC    logLik   LR.stat df  Pr(>Chisq)   
 m1      1  4473.9 -2236.0                        
 m2      9  4187.3 -2084.7  302.62  8   < 2.2e-16 ***

​I am used to comparing these kinds of models using chi-squared values, a chi-squared difference, and a chi-squared difference test. Since all other models in the paper are compared this way, and since I'd like to report them in a table together: why exactly is this model test different from my other model tests in which I get chi-squared values and difference tests? Can I obtain chi-squared values from this test?

Results from other model comparisons (e.g., GLMER), look like this: 

    #Df AIC     BIC     logLik  Chisq   Chi     DF diff Pr(>Chisq)
m3  13  11288   11393   -5630.9 392.16          
m4  21  11212   11382   -5584.9 92.02   300.14  8       0.001
Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ To what "paper" do you refer? $\endgroup$
    – whuber ♦
    Commented Sep 11, 2013 at 17:26

1 Answer 1

Reset to default
13
$\begingroup$

The "chi-square value" you're looking for is the deviance (-2*(log likelihood), at least up to an additive constant that doesn't matter for the purposes of inference. R gives you the log-likelihood above (logLik) and the likelihood ratio statistic (LR.stat): the LR stat is twice the difference in the log-likelihoods (2*(2236.0-2084.7)).

I'm a little puzzled by your anova results, since they don't seem to match the format that I get when I run anova() on two glm() fits: in particular, stats:::anova.glm (the anova method for glm objects) doesn't print the AIC ... are m1 and m2 really glm objects? (e.g. try class("m1"))

Can you give us a reproducible example? Here's what I get for a simple example, modified from ?glm:

## Dobson (1990) Page 93: Randomized Controlled Trial :
d.AD <- data.frame(counts=c(18,17,15,20,10,20,25,13,12),
                   outcome=gl(3,1,9),
                   treatment=gl(3,3))
glm1 <- glm(counts ~ outcome + treatment, family = poisson, data=d.AD)
glm0 <- update(glm1, . ~ 1)

Model comparison gives the residual deviances (your "chi-squared value") and the differences between them ...

anova(glm0,glm1,test="Chisq")
## Analysis of Deviance Table
## 
## Model 1: counts ~ 1
## Model 2: counts ~ outcome + treatment
##   Resid. Df Resid. Dev Df    Deviance Pr(>Chi)
## 1         8    10.5814                     
## 2         4     5.1291  4      5.4523    0.244

(if I left out the test="Chisq", I would get all of the above but without the p-value)

I see from your cross-posting at http://article.gmane.org/gmane.comp.lang.r.general/299377 that you're actually using ordinal::clm, in which case we do get output that looks like what you have above (it's important to be precise) ... the results are not identical because the model is slightly different, and you are given the log-likelihoods (=-deviance/2) rather than the deviances, but the difference between the deviances ("LR.stat") is similar.

library(ordinal)
clm1 <- clm(ordered(counts) ~
   outcome + treatment, family = poisson, data=d.AD)
clm0 <- update(clm1, . ~ 1)
anova(clm0,clm1)
##       no.par      AIC  logLik LR.stat df Pr(>Chisq)
## clm0      7    50.777 -18.389                      
## clm1     11    52.839 -15.419  5.9389  4     0.2038
Improve this answer
$\endgroup$
5
  • $\begingroup$ Ben, thank you for this very helpful comment! You are right, I confused CLM and GLM -- I am using GLMs now, and get the output you describe above. Residual Deviances in my case are M1=9693 (resid. DF=12339) and M2=10945 (resid. DF=12347) (Deviance=-1252) with DF_diff=-8 (p<0.001). Does that mean I can report these two values as chi-square values in a paper, or would that be incorrect? $\endgroup$
    – Torvon
    Commented Sep 12, 2013 at 11:47
  • 1
    $\begingroup$ You can report them as chi-square values, but it would generally be more informative to report them as deviances; there are lots of different statistics that are chi-square distributed $\endgroup$
    – Ben Bolker
    Commented Sep 12, 2013 at 13:18
  • $\begingroup$ Thank you again. Although deviances are compared, is this a "chi-square difference test"? Would you know of any example where such a model is reported in a paper? I'm used to reporting chi-square, df, AIC (rmsea etc.), chi-square-diff and df-diff, and p. Not sure what to report in this case above. Thank you! $\endgroup$
    – Torvon
    Commented Sep 23, 2013 at 14:28
  • 2
    $\begingroup$ If you are working in a GLM(M) context then "chi-square" and "deviance" are synonymous as far as I can tell (but as I pointed out "deviance" is more precise). $\endgroup$
    – Ben Bolker
    Commented Sep 23, 2013 at 14:31
  • $\begingroup$ Last question: usually in my analyses the DF are 8 smaller in the more constrained model. In this particular case above, using the code you suggested, "residual df" of the freely estimated model m1 is 8 smaller than the constrained model m2 (12339 vs. 12347), although the fit is much better for m1 (deviance: 9693 vs 10945). How can that be? Thank you! $\endgroup$
    – Torvon
    Commented Sep 25, 2013 at 14:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.