3
$\begingroup$

When looking at a glm with non categorical predictors I am given to understand that the intercept the the predicted value of your measure when all predictor variables are at 0.

This therefore means that when looking at the coefficients of such a glm we take the estimate to be the ratio change of the measure with a 1 unit increase in the predictor variable. The p-value associated with this then shows is this change is significant enough for that predictor variable to have an effect on the models predictive power?

However when we look at a glm with categorical variables the intercept is the value of your measure when all predictor variables are at their first factor level? How do I then interpret the p-values associated with these coefficients?

Here is an example model:

Call:
glm(formula = count ~ origin + variable + origin * variable, 
    family = "poisson", data = count_filt_FGT_free)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-1.6877  -0.6963  -0.3758   0.0306   5.1953  

Coefficients:
                                 Estimate Std. Error z value Pr(>|z|)    
(Intercept)                      0.217065   0.110432   1.966   0.0493 *  
originfree                      -0.247836   0.166794  -1.486   0.1373    
variableDuplication              0.136576   0.151107   0.904   0.3661    
variableKnown_target            -1.634130   0.273254  -5.980 2.23e-09 ***
variablePhylogeny                0.125880   0.151485   0.831   0.4060    
originfree:variableDuplication   0.008606   0.227974   0.038   0.9699    
originfree:variableKnown_target  0.040197   0.408914   0.098   0.9217    
originfree:variablePhylogeny     0.005696   0.228629   0.025   0.9801 

The intercept is made up of the first factor level of origin (FGT) and variable (proximity). So when looking at the exp of originfree estimate we see that the count changes by a ratio of exp(-0.247836) = 0.7804879. Does the p-value associated with this (0.1373) show that for variableProximity there is no significant difference when being originFree ?

$\endgroup$

1 Answer 1

2
$\begingroup$

The intercept is made up of the first factor level of origin (free)

This appears to be incorrect because of:

originfree                      -0.247836   0.166794  -1.486   0.1373 

if free was the reference level for origin this line would start with originXYZ where XYZ is the other level.

when looking at the exp of originfree estimate we see that the count changes by a ratio of exp(-0.247836) = 0.7804879.

This is also incorrect. The variable origin is involved in an interaction with variable so the main effect is conditional on variable being at it's reference level, so you can just add "when variable is at it's reference level" to that sentence.

Does the p-value associated with this (0.1373) show that for variableProximity there is no significant difference when being originFree ?

Not quite. It should be interpreted as: If there is actually no difference in the outcome between origin "free" and it's reference level, when variable is at it's reference level, then the probability of obtaining -0.247836 (or less) again, is 0.1373.

Try to avoid statements about "significant differences" because they depend on arbitrary thresholds for p values - if you obtained a p value of 0.0999999 someone might say that there is a significant difference, whereas if the p value was 0.1000001 the same person might say there is no significant difference (at the 0.1 level). I would say that the results are the same. Also, if a different person had a "bible" that told them 0.05 was the "correct significance level" they would say both results are not significant, whereas the person who's bible told them to use 0.15 would conclude that both are significant. I hope you can see how arbitrary this is.

$\endgroup$
6
  • $\begingroup$ Ah yeh my bad I mean the intercept is made up of originat its reference level, edited to change now. But thank you for clarifying how I should be looking at the p values. So the p values associated with coefficients are not really meant to be used in saying X or Y is important in the model? But rather that variableknown_target has evidence suggesting estimates of count associated with is when origin is at it's reference level are different from variableat its reference level (p= 2.23e-09) ? $\endgroup$
    – Lamma
    Jul 31, 2020 at 7:16
  • $\begingroup$ For variable you can say that there is (strong) evidence that the oucome is different between known_target and the reference level, when origin is at it's reference level. But note that there is almost no evidence of an interaction so unless you have a priori reason to believe there should be an interation, then take it out and it will make your life easier with interpretation of the main effects (because when there is no interaction the main effects are not conditional) $\endgroup$ Jul 31, 2020 at 8:08
  • $\begingroup$ @RovertLong My reason for including it is as you say I expected there to be an interaction as origin is just where the sample came from and variable are the different categories of things I am counting. So if there is a difference in origin I expected that difference to be reflected in variable Would this then lead us to believe there is something else wrong with the model? $\endgroup$
    – Lamma
    Jul 31, 2020 at 8:34
  • $\begingroup$ It's a bit difficult to say. Have you plotted the residuals ? Are they approximately normally distributed ? $\endgroup$ Jul 31, 2020 at 9:28
  • $\begingroup$ I am not entirely sure how to plot the residuals so have included the code I used to do it along with the plot above. It is wrong of me to include variable in the model if it is itself the categories things I am counting? $\endgroup$
    – Lamma
    Jul 31, 2020 at 9:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.