3
$\begingroup$

I'm trying to test the fit of two logistic regression models. I understand that deviance and AIC are good methods for this, but what if one model has better AIC but worse deviance? This is what I think has happened in my case.

Here's the summary/anova table from the first logistic regression:

Call:
glm(formula = winperc ~ prevwinperc, family = binomial(link = "logit"), 
    data = New_Data, weights = gp_col)

Deviance Residuals: 
   Min       1Q   Median       3Q      Max  
-4.9750  -1.0090   0.0503   1.0859   3.7078  

Coefficients:

           Estimate Std. Error z value Pr(>|z|)    
 (Intercept) -1.02740    0.07812  -13.15   <2e-16 
  prevwinperc  2.04401    0.15453   13.23   <2e-16 

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 710.52  on 235  degrees of freedom
 Residual deviance: 534.69  on 234  degrees of freedom
 AIC: 1842.8

Number of Fisher Scoring iterations: 3

Analysis of Deviance Table    
Model: binomial, link: logit    
Response: winperc    
Terms added sequentially (first to last)

          Df Deviance Resid. Df Resid. Dev
NULL                          235     710.52
prevwinperc  1   175.83       234     534.69

and here's the same info for the second:

Call:
glm(formula = winperc ~ prevPyth, family = binomial(link = "logit"), 
    data = New_Data, weights = gp_col)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-4.9997  -1.1134   0.0745   1.1310   3.7486  

Coefficients:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept) -1.13383    0.08564  -13.24   <2e-16 ***
prevPyth     2.25682    0.16969   13.30   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 710.52  on 235  degrees of freedom
Residual deviance: 532.75  on 234  degrees of freedom
AIC: 1840.8

Number of Fisher Scoring iterations: 3

Analysis of Deviance Table    
Model: binomial, link: logit    
Response: winperc    
Terms added sequentially (first to last)  

         Df Deviance Resid. Df Resid. Dev
NULL                       235     710.52
prevPyth  1   177.77       234     532.75

So the first model has higher AIC but lower Deviance.

$\endgroup$
4
  • 2
    $\begingroup$ Oh, man, that's a mess. I will try to arrange it better. $\endgroup$ – mike Mar 9 '12 at 18:22
  • 3
    $\begingroup$ How so? In model 1 AIC is 1843 and deviance is 535. In model 2, AIC is 1841 and deviance is 533. Model 2 is better by both measures. $\endgroup$ – Peter Flom Mar 9 '12 at 18:50
  • $\begingroup$ Oh. Thanks. So the deviance of interest is "Residual Deviance". Then what is the 177.77 number labeled "Deviance" in the second anova table? $\endgroup$ – mike Mar 9 '12 at 19:37
  • 3
    $\begingroup$ It's the "Explained deviance" due to the term on the left. I wish it was labeled something like "Explained deviance", myself. $\endgroup$ – jbowman Mar 9 '12 at 20:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.