Please help! I have recently been criticized for using pairwise comparisons to explain all three levels of a factor within a negative binomial GLM rather than all levels at once. I was told that it is "long-winded" and "uneccessary". I was under the impression that in GLMs one cannot bulk all levels of a factor together to obtain a test statistic and corresponding p-value.

Obviously if a factor is "insignificant" at any level then carrying out a post-hoc analysis is pointless. My levels all have there own p-values therefore I discussed these values from the below global model. I was told to do an ANOVA instead which I don't believe is suitable for overdispersed, zero-inflated data.

p-value for all levels of a factor anyone?

(Below, lower field layer 0, upper field layer 1 and change1 is in intercept)

    Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
   -2.4284  -0.7956  -0.3862   0.4045   2.4233  

                            Estimate Std. Error z value Pr(>|z|)   
    (Intercept)                    4.3410884  1.8219786   2.383  0.01719 * 
    Height                         0.0373584  0.0119929   3.115  0.00184 **
    Width                         -0.0007891  0.0008246  -0.957  0.33859   
    MeanMin                       -0.1731877  0.1404434  -1.233  0.21752   
    as.factor(Site_Treat)2        -0.4080256  0.2480438  -1.645  0.09998 . 
    as.factor(Change)2            -0.4940398  0.1755487  -2.814  0.00489 **
    as.factor(Change)3            -0.1613766  0.1763677  -0.915  0.36019   
    as.factor(Lower_Field_Layer)1  0.4873488  0.2931585   1.662  0.09643 . 
    as.factor(Lower_Field_Layer)2 -0.3292409  0.3717863  -0.886  0.37585   
    as.factor(Upper_Field_Layer)2 -0.0081040  0.3257734  -0.025  0.98015   
    Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

   (Dispersion parameter for Negative Binomial(4.7795) family taken to be 1)

    Null deviance: 96.392  on 46  degrees of freedom
    Residual deviance: 47.968  on 37  degrees of freedom
    AIC: 403.94

Best wishes, Platypezid


You could fit the model with and without the categorical variable and do a likelihood ratio test to determine it's overall significance.

  • $\begingroup$ Ah yes, Macro mentioned this last year to me on here. He explained it wonderfully well to me on the question "Checking if a nominal variable is important in a GLM model". I found the log(lik) method hard to perform, however, in R. Is there anywhere that you would recommend trying to perform this test. Most of my analysis is using Brodgar which lacks this test. My data has lots of zeros and the data is count data of insects that I trapped. Best wishes! $\endgroup$ – Platypezid Jan 25 '12 at 15:42
  • $\begingroup$ A few more Qs: 1) Do you get a p-value from a log likelihood test. 2) If the factor is insignificant, is it then okay to read p-values from other factors within the global model while this (multi-level) factor is still in the model? $\endgroup$ – Platypezid Jan 25 '12 at 15:48
  • $\begingroup$ Yes, a likelihood test produces a p-value from a chi-square test (by default). see stat.ethz.ch/R-manual/R-patched/library/MASS/html/… for an example. $\endgroup$ – B_Miner Jan 25 '12 at 16:10
  • $\begingroup$ Great thanks! and then if it is insignificant will I remove it from the model and read p-values or is it okay to keep it in the model? I presume it is unimportant so remove.... This is what bugs me. Everything hinges on everything else in nature so I'm always afraid to tamper $\endgroup$ – Platypezid Jan 25 '12 at 16:13
  • $\begingroup$ As long as there is not strong correlation between the predictors, I personally don't think there are issues with p-values for other variables simply because a determined non-significant factor is retained in the model. Significance tests (LR, Wald etc) are testing the improvement of a model fit for a given variable given the model (i.e. controlling for other predictors). $\endgroup$ – B_Miner Jan 25 '12 at 16:15

You have probably already solved this, but I had the same problem and found a solution in "A Beginner's Guide to GLM and GLMM with R".

If you use the drop1() function you get one P-value for each factor in your model (not for each level of the factor.

This is a similar approach to the one suggested by B_Miner, but for me a lot easier to calculate.

  • $\begingroup$ It's exactly the same to B_Miner's solution $\endgroup$ – Firebug Feb 14 '19 at 12:15
  • $\begingroup$ It just wasn't clear to me from their solution how to simply do this. (But this may just be me.) $\endgroup$ – bwb Feb 14 '19 at 13:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for?Browse other questions tagged or ask your own question.