I have significant differences detected in my glmer model summary but not in my mcp (Tukey) multiple comparisons summary. Why?

Question

I am working on a project looking at ant recruitment to bait in the lab. I want to see which baits ('treatment') the ants recruit to more/less and have multiple measurements for each colony over the course of multiple weeks. I have (hopefully) accounted for varience between colonies and each measurement collection by nesting 'colony' in 'DAT' (days after treatment). My data was overdispersed so I added an observation level random factor to correct for this ('obs'). I am using the lme4 package for the overall glmer model with results below

MODEL

> mod2015<-glmer(numants~Treatment+(1|colony)+(1|DAT)+(1|obs), family=poisson, data=ncl2015)
> summary(mod2015)

Generalized linear mixed model fit by maximum
  likelihood (Laplace Approximation) [glmerMod]
 Family: poisson  ( log )
Formula: 
numants ~ Treatment + (1 | colony) + (1 | DAT) + (1 | obs)
   Data: ncl2015

     AIC      BIC   logLik deviance df.resid 
  1281.8   1302.6   -633.9   1267.8      137 

Scaled residuals: 
     Min       1Q   Median       3Q      Max 
-2.82084 -0.26069  0.02638  0.15502  0.49829 

Random effects:
 Groups Name        Variance Std.Dev.
 obs    (Intercept) 0.1954   0.4420  
 colony (Intercept) 0.1519   0.3897  
 DAT    (Intercept) 0.1275   0.3571  
Number of obs: 144, groups:  
obs, 144; colony, 24; DAT, 6

Fixed effects:
             Estimate Std. Error z value Pr(>|z|)    
(Intercept)    3.1883     0.2310  13.801   <2e-16 ***
TreatmentHON   0.5283     0.2522   2.095   0.0362 *  
TreatmentKB    0.5512     0.2522   2.186   0.0288 *  
TreatmentPB    0.1277     0.2528   0.505   0.6136    
---
Signif. codes:  
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
            (Intr) TrtHON TrtmKB
TreatmntHON -0.551              
TreatmentKB -0.551  0.504       
TreatmentPB -0.549  0.503  0.503

and am using the multcomp package to conduct pairwise comparisons using the glht function with the results below

POST HOC

> mcpncl2015<-glht(mod2015, mcp(Treatment="Tukey"))
> summary(mcpncl2015)

     Simultaneous Tests for General Linear Hypotheses

Multiple Comparisons of Means: Tukey Contrasts


Fit: glmer(formula = numants ~ Treatment + (1 | colony) + (1 | DAT) + 
    (1 | obs), data = ncl2015, family = poisson)

Linear Hypotheses:
                Estimate Std. Error z value Pr(>|z|)
HON - BUFF == 0  0.52830    0.25223   2.095    0.155
KB - BUFF == 0   0.55117    0.25218   2.186    0.127
PB - BUFF == 0   0.12769    0.25285   0.505    0.958
KB - HON == 0    0.02287    0.25109   0.091    1.000
PB - HON == 0   -0.40061    0.25179  -1.591    0.384
PB - KB == 0    -0.42348    0.25173  -1.682    0.333
(Adjusted p values reported -- single-step method)

PROBLEM As you can see, I get significant differences (p values) when from the over all glmer model summary but those significant differences are not detected when I run the glht function for multiple comparisons using Tukey post hoc test. Can anyone tell me why and how to remedy this? I hope I've explained the issue well enough for help.

Answer 1

BOTH glmer and mcp (Tukey) results show that the treatment HON and treatment KB have a significant effect according to the z-values. The two methods glmer and mcp Tukey method test for significance in different ways. Your interpretation is not correct with regard to mcp (Tukey).

Answer 2

The results are quite consistent.

If you looked at each variable in isolation and had pre-specified that one variable to look at and had pre-specified that all the other variables are in the model, then you can in a null-hypothesis testing framework directly use the p-values from glmer and e.g. HON would only have a coefficient this significant under the null hypothesis (and assuming your model is right) 3.6% of the time.

If you are looking at multiple variables, then you have a multiple comparisons problems and you would expect to get some (i.e. for at least one of the many variables you look at) unadjusted p-value this low over 5% of the time under the null hypothesis.