4
$\begingroup$

Please help! I have recently been criticized for using pairwise comparisons to explain all three levels of a factor within a negative binomial GLM rather than all levels at once. I was told that it is "long-winded" and "uneccessary". I was under the impression that in GLMs one cannot bulk all levels of a factor together to obtain a test statistic and corresponding p-value.

Obviously if a factor is "insignificant" at any level then carrying out a post-hoc analysis is pointless. My levels all have there own p-values therefore I discussed these values from the below global model. I was told to do an ANOVA instead which I don't believe is suitable for overdispersed, zero-inflated data.

p-value for all levels of a factor anyone?

(Below, lower field layer 0, upper field layer 1 and change1 is in intercept)

    Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
   -2.4284  -0.7956  -0.3862   0.4045   2.4233  

    Coefficients:
                            Estimate Std. Error z value Pr(>|z|)   
    (Intercept)                    4.3410884  1.8219786   2.383  0.01719 * 
    Height                         0.0373584  0.0119929   3.115  0.00184 **
    Width                         -0.0007891  0.0008246  -0.957  0.33859   
    MeanMin                       -0.1731877  0.1404434  -1.233  0.21752   
    as.factor(Site_Treat)2        -0.4080256  0.2480438  -1.645  0.09998 . 
    as.factor(Change)2            -0.4940398  0.1755487  -2.814  0.00489 **
    as.factor(Change)3            -0.1613766  0.1763677  -0.915  0.36019   
    as.factor(Lower_Field_Layer)1  0.4873488  0.2931585   1.662  0.09643 . 
    as.factor(Lower_Field_Layer)2 -0.3292409  0.3717863  -0.886  0.37585   
    as.factor(Upper_Field_Layer)2 -0.0081040  0.3257734  -0.025  0.98015   
    ---
    Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 

   (Dispersion parameter for Negative Binomial(4.7795) family taken to be 1)

    Null deviance: 96.392  on 46  degrees of freedom
    Residual deviance: 47.968  on 37  degrees of freedom
    AIC: 403.94

Best wishes, Platypezid

$\endgroup$
1
  • $\begingroup$ Unfortunately in statistics and science in general there is sometimes the actual way it should be done and the way people do it because it looks good to others. p-values are often a notorious example of this, there have been calls for p-values to be done away with altogether. There is no point in having a p-value if you don't know where it comes from, especially when you already have the pairwise values and need to explain them. $\endgroup$
    – Padraig
    Oct 14, 2023 at 21:49

2 Answers 2

2
$\begingroup$

You could fit the model with and without the categorical variable and do a likelihood ratio test to determine it's overall significance.

$\endgroup$
6
  • $\begingroup$ Ah yes, Macro mentioned this last year to me on here. He explained it wonderfully well to me on the question "Checking if a nominal variable is important in a GLM model". I found the log(lik) method hard to perform, however, in R. Is there anywhere that you would recommend trying to perform this test. Most of my analysis is using Brodgar which lacks this test. My data has lots of zeros and the data is count data of insects that I trapped. Best wishes! $\endgroup$
    – Platypezid
    Jan 25, 2012 at 15:42
  • $\begingroup$ A few more Qs: 1) Do you get a p-value from a log likelihood test. 2) If the factor is insignificant, is it then okay to read p-values from other factors within the global model while this (multi-level) factor is still in the model? $\endgroup$
    – Platypezid
    Jan 25, 2012 at 15:48
  • $\begingroup$ Yes, a likelihood test produces a p-value from a chi-square test (by default). see stat.ethz.ch/R-manual/R-patched/library/MASS/html/… for an example. $\endgroup$
    – B_Miner
    Jan 25, 2012 at 16:10
  • $\begingroup$ Great thanks! and then if it is insignificant will I remove it from the model and read p-values or is it okay to keep it in the model? I presume it is unimportant so remove.... This is what bugs me. Everything hinges on everything else in nature so I'm always afraid to tamper $\endgroup$
    – Platypezid
    Jan 25, 2012 at 16:13
  • $\begingroup$ As long as there is not strong correlation between the predictors, I personally don't think there are issues with p-values for other variables simply because a determined non-significant factor is retained in the model. Significance tests (LR, Wald etc) are testing the improvement of a model fit for a given variable given the model (i.e. controlling for other predictors). $\endgroup$
    – B_Miner
    Jan 25, 2012 at 16:15
0
$\begingroup$

You have probably already solved this, but I had the same problem and found a solution in "A Beginner's Guide to GLM and GLMM with R".

If you use the drop1() function you get one P-value for each factor in your model (not for each level of the factor.

This is a similar approach to the one suggested by B_Miner, but for me a lot easier to calculate.

$\endgroup$
2
  • $\begingroup$ It's exactly the same to B_Miner's solution $\endgroup$
    – Firebug
    Feb 14, 2019 at 12:15
  • $\begingroup$ It just wasn't clear to me from their solution how to simply do this. (But this may just be me.) $\endgroup$
    – bwb
    Feb 14, 2019 at 13:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.