3
$\begingroup$

I've data df and i'm wondering why p-value change according the order of the independent variables product and age ?

How to know which one to place firstly ? 

     df=structure(list(Age = structure(c(1L, 1L, 2L, 2L, 2L, 2L, 2L, 
        2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 
        3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 
        4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 
        4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 5L, 5L, 5L, 5L, 5L, 5L, 
        5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 6L, 6L, 6L, 
        6L, 6L, 6L, 6L, 6L, 6L, 6L, 6L, 6L, 6L, 6L, 6L, 6L, 6L, 6L, 7L, 
        7L, 7L, 7L, 7L, 7L, 7L, 7L, 7L, 7L, 7L, 7L, 7L, 7L, 7L, 7L, 7L, 
        7L, 7L, 7L, 7L, 7L, 8L, 8L, 8L, 8L, 8L, 8L, 8L, 8L, 8L, 8L, 8L, 
        8L, 8L, 8L, 8L, 8L, 8L, 8L, 8L, 8L, 8L, 8L, 8L, 8L, 8L, 9L, 9L, 
        9L, 9L, 9L, 9L, 9L, 9L, 9L, 9L, 9L, 9L, 9L, 9L, 9L, 9L, 9L, 9L, 
        9L, 9L, 9L, 9L, 9L, 10L, 10L, 10L, 10L, 10L, 10L, 10L, 10L, 10L, 
        10L, 10L, 10L, 10L, 10L, 10L, 10L, 10L, 10L, 10L, 10L, 10L, 11L, 
        11L, 11L, 11L, 11L, 11L, 11L, 12L, 13L, NA), .Label = c("(10,15]", 
        "(15,20]", "(20,25]", "(25,30]", "(30,35]", "(35,40]", "(40,45]", 
        "(45,50]", "(50,55]", "(55,60]", "(60,65]", "(65,70]", "(70,75]", 
        "(75,80]"), class = "factor"), Product = c(6L, 16L, 4L, 6L, 9L, 
        11L, 14L, 15L, 17L, 20L, 23L, 26L, 28L, 30L, 32L, 33L, 36L, 44L, 
        47L, 4L, 6L, 9L, 11L, 14L, 15L, 17L, 18L, 20L, 23L, 25L, 26L, 
        28L, 30L, 32L, 33L, 36L, 44L, 50L, 51L, 4L, 6L, 9L, 11L, 14L, 
        15L, 16L, 17L, 18L, 20L, 22L, 23L, 26L, 28L, 29L, 30L, 32L, 33L, 
        36L, 37L, 39L, 43L, 44L, 50L, 51L, 58L, 4L, 6L, 9L, 11L, 14L, 
        15L, 17L, 18L, 20L, 23L, 26L, 28L, 30L, 32L, 33L, 36L, 37L, 47L, 
        58L, 4L, 6L, 9L, 11L, 14L, 15L, 17L, 18L, 20L, 23L, 25L, 26L, 
        28L, 30L, 33L, 35L, 36L, 44L, 4L, 6L, 9L, 14L, 15L, 17L, 20L, 
        22L, 23L, 25L, 26L, 28L, 29L, 30L, 33L, 36L, 38L, 43L, 44L, 50L, 
        51L, 58L, 4L, 6L, 9L, 11L, 14L, 15L, 16L, 17L, 20L, 23L, 25L, 
        26L, 28L, 30L, 32L, 33L, 35L, 36L, 37L, 43L, 44L, 50L, 51L, 52L, 
        58L, 4L, 6L, 9L, 11L, 14L, 15L, 16L, 17L, 18L, 20L, 23L, 26L, 
        28L, 29L, 30L, 32L, 33L, 36L, 43L, 44L, 50L, 51L, 52L, 4L, 6L, 
        9L, 14L, 17L, 18L, 20L, 23L, 25L, 26L, 28L, 30L, 32L, 33L, 35L, 
        36L, 43L, 44L, 51L, 52L, 58L, 6L, 9L, 20L, 23L, 26L, 28L, 36L, 
        4L, 15L, 6L), Yes = c(2L, 1L, 4L, 5L, 2L, 1L, 2L, 2L, 2L, 2L, 
        1L, 3L, 4L, 4L, 1L, 4L, 1L, 1L, 1L, 6L, 2L, 4L, 4L, 8L, 1L, 4L, 
        1L, 1L, 3L, 1L, 5L, 1L, 1L, 0L, 2L, 1L, 1L, 1L, 2L, 4L, 4L, 8L, 
        2L, 6L, 1L, 1L, 4L, 4L, 2L, 1L, 9L, 3L, 3L, 1L, 2L, 1L, 2L, 2L, 
        1L, 1L, 1L, 1L, 1L, 3L, 1L, 4L, 3L, 13L, 2L, 1L, 2L, 4L, 3L, 
        2L, 3L, 2L, 5L, 2L, 1L, 1L, 1L, 1L, 1L, 2L, 4L, 9L, 9L, 4L, 5L, 
        1L, 8L, 2L, 1L, 3L, 1L, 0L, 3L, 1L, 1L, 1L, 2L, 2L, 3L, 7L, 3L, 
        2L, 1L, 7L, 3L, 1L, 1L, 1L, 3L, 8L, 1L, 2L, 1L, 3L, 1L, 1L, 2L, 
        1L, 2L, 2L, 2L, 3L, 2L, 3L, 2L, 1L, 2L, 3L, 2L, 2L, 1L, 3L, 3L, 
        6L, 0L, 0L, 1L, 0L, 1L, 1L, 3L, 1L, 1L, 1L, 1L, 2L, 3L, 8L, 2L, 
        6L, 1L, 1L, 1L, 3L, 3L, 7L, 5L, 4L, 1L, 2L, 1L, 2L, 3L, 3L, 3L, 
        3L, 1L, 0L, 1L, 3L, 6L, 3L, 3L, 4L, 2L, 3L, 1L, 2L, 1L, 2L, 1L, 
        2L, 0L, 2L, 1L, 2L, 2L, 1L, 1L, 1L, 0L, 3L, 2L, 0L, 2L, 1L, 1L, 
        1L, 1L), No = c(0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 
        1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 1L, 0L, 0L, 0L, 0L, 
        0L, 0L, 0L, 0L, 1L, 2L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 
        0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 
        0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 1L, 0L, 
        0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 2L, 0L, 1L, 0L, 
        0L, 0L, 0L, 1L, 2L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 1L, 1L, 
        0L, 0L, 3L, 0L, 1L, 1L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 
        0L, 0L, 2L, 0L, 0L, 0L, 1L, 1L, 0L, 1L, 0L, 0L, 3L, 0L, 1L, 1L, 
        0L, 1L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 
        0L, 1L, 0L, 3L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 1L, 0L, 0L, 1L, 
        0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 1L, 
        0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 0L)), class = "data.frame", row.names = c(NA, 
        -203L), .Names = c("Age", "Product", "Yes", "No"))

        attach(df)

       anova(glm(cbind(Yes,No) ~ Age + Product, family=binomial),test="Chisq")
Analysis of Deviance Table

Model: binomial, link: logit

Response: cbind(Yes, No)

Terms added sequentially (first to last)


        Df Deviance Resid. Df Resid. Dev Pr(>Chi)  
NULL                      201     176.98           
Age     12  24.4260       189     152.56  0.01779 *
Product  1   1.8942       188     150.66  0.16873  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>         anova(glm(cbind(Yes,No) ~ Product + Age, family=binomial),test="Chisq")
Analysis of Deviance Table

Model: binomial, link: logit

Response: cbind(Yes, No)

Terms added sequentially (first to last)


        Df Deviance Resid. Df Resid. Dev Pr(>Chi)  
NULL                      201     176.98           
Product  1   2.9839       200     174.00   0.0841 .
Age     12  23.3364       188     150.66   0.0250 *
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Improve this question
$\endgroup$
1
  • $\begingroup$ This issue is covered in several other places on this site, including this page. Please look at that page and the links from it, and revise your question to focus on what you still don't understand thereafter. Otherwise this is likely to be closed as a duplicate question. $\endgroup$
    – EdM
    Commented Jan 18, 2018 at 15:31

1 Answer 1

Reset to default
4
$\begingroup$

The hypothesis tests that are being performed by the call to anova() are based on adding terms to your model sequentially and asking whether or not each added term improves the model given all of those that came before it. Hence, a different variable order will result in different hypothesis tests being performed.

For the first table, you begin will the Null model--i.e. the model without any independent variables--which has a residual deviance of 176.98. Upon adding the variable Age to your model, the residual deviance drops to 152.56 and a chi-squared test gives a p-value of 0.01779. That is, the addition of the variable Age has arguably improved the fit when compared to the Null model. The third line in the table now says given that Age is already in the model, what improvement is gained by additionally adding Product to the model? In that case, the drop in the deviance was not large enough to yield a significant p-value.

For the second table, we proceed similarly by beginning with the Null model. The first variable added in this case is Product. Hence, the first p-value of 0.0841 comes from comparing the Null model to the model with the independent variable Product. The third line in the table then asks, does the addition of the variable Age given that we already have the variable Product in the model improve the fit? In this case, one might argue that 0.025 is small enough to declare that adding Age does, in fact, improve the model.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank so much for, this,I was stack on it. I understand better. For research purpose, Should I place firstly the variable that is more susceptible to be more implicated and more have impact on the model ? or is it "ethic" to taket he combination that enhance the p-value of each variables, sometimes significance appears just by reversing the order. $\endgroup$
    – ranell
    Commented Jan 18, 2018 at 15:56
  • 1
    $\begingroup$ There's more than one way to proceed. You could, in general, compare all four possible models--i.e. Null, only Age, only Product, both Age and Product--with some model selection criterion like AIC or BIC to determine the "best". Given specifically the two tables above, I would conclude that Age is a good predictor whereas Product is not. Mainly, because the p-value for Product alone is only 0.0841, which doesn't seem very significant. Of course, many other behaviours are possible especially when a large number of regressors are included in the model. $\endgroup$
    – Adam B Kashlak
    Commented Jan 18, 2018 at 16:59
  • 2
    $\begingroup$ The dependence on the order of variables comes from the choice of Type I ANOVA implicit in the anova() function in R. There are other Types of ANOVA, as explained for example on this page, for which the order of variables in the model does not matter. Those alternatives, however, have other potential downsides. So it's not always an easy choice. Whatever choice about type of ANOVA is made, it's crucial to explain in your writing why the choice was made. $\endgroup$
    – EdM
    Commented Jan 18, 2018 at 17:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.