# Conflicting results between GLMM and Post-hoc lsmeans

I'm studying the effect of pH and cross-types on mortality of fish. Treatment is categorical (2 levels: control and low pH) and cross-types is also categorical (4 levels: parents wild male x wild female (WMWF), wild male x farmed female (WMFF), farmed male x wild female (FMWF), and farmed male x farmed female (FMFF)). There was 6 tanks in total (3 control and 3 at low pH) and each tank had 15 fish of each cross-type (60 fish total/tank). Since mortality is a count and that there was higher mortality in one of the control tank, I used Poisson GLMM to account for the tank effect.

Here's the model and summary results:

Generalized linear mixed model fit by maximum likelihood (Laplace Approximation) ['glmerMod']
Family: poisson  ( log )
Formula: mortality.count ~ Tr * Cross + (1 | Tank)
Data: pHdat

AIC      BIC   logLik deviance df.resid
93.8    104.4    -37.9     75.8       15

Scaled residuals:
Min      1Q  Median      3Q     Max
-1.1311 -0.4171 -0.2554  0.1608  1.2889

Random effects:
Groups Name        Variance Std.Dev.
Tank   (Intercept) 2.225    1.492
Number of obs: 24, groups:  Tank, 6

Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept)       -1.666e+00  1.377e+00  -1.210   0.2264
TrLOWpH            3.053e+00  1.647e+00   1.854   0.0637 .
CrossFMWF          9.810e-01  6.770e-01   1.449   0.1473
CrossWMFF          9.810e-01  6.770e-01   1.449   0.1474
CrossWMWF          2.248e-05  8.165e-01   0.000   1.0000
TrLOWpH:CrossFMWF -1.754e+00  8.378e-01  -2.094   0.0363 *
TrLOWpH:CrossWMFF -1.243e+00  7.970e-01  -1.560   0.1188
TrLOWpH:CrossWMWF -6.190e-01  9.415e-01  -0.658   0.5109
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
(Intr) TrLOWH CrFMWF CrWMFF CrWMWF TLOWH:CF TLOWH:CWMF
TrLOWpH     -0.835
CrossFMWF   -0.358  0.299
CrossWMFF   -0.358  0.299  0.727
CrossWMWF   -0.296  0.248  0.603  0.603
TLOWH:CFMWF  0.289 -0.297 -0.808 -0.588 -0.487
TLOWH:CWMFF  0.304 -0.313 -0.618 -0.849 -0.512  0.614
TLOWH:CWMWF  0.257 -0.265 -0.523 -0.523 -0.867  0.520    0.547


As you can see, the fish in low pH tanks of the cross FMWF died (weakly but still) significantly less than the baseline (FMFF).

Now I wanted to see if there was significant differences between the cross for each treatment (not only compared to the baseline) so I used lsmeans. Here's the results:

lsmeans(glmm.0,pairwise~Tr*Cross,adjust="tukey")
$lsmeans Tr Cross lsmean SE df asymp.LCL asymp.UCL CTRL FMFF -1.6658411 1.3770371 NA -4.3647842 1.033102 LOWpH FMFF 1.3873820 0.9065822 NA -0.3894865 3.164251 CTRL FMWF -0.6848201 1.2991734 NA -3.2311531 1.861513 LOWpH FMWF 0.6141089 0.9548037 NA -1.2572719 2.485490 CTRL WMFF -0.6848677 1.2991756 NA -3.2312050 1.861470 LOWpH WMFF 1.1251162 0.9192163 NA -0.6765147 2.926747 CTRL WMWF -1.6658186 1.3770335 NA -4.3647547 1.033117 LOWpH WMWF 0.7683761 0.9422428 NA -1.0783859 2.615138 Results are given on the log (not the response) scale. Confidence level used: 0.95$contrasts
contrast                     estimate        SE df z.ratio p.value
CTRL,FMFF - LOWpH,FMFF  -3.053223e+00 1.6467764 NA  -1.854  0.5828
CTRL,FMFF - CTRL,FMWF   -9.810210e-01 0.6770180 NA  -1.449  0.8342
CTRL,FMFF - LOWpH,FMWF  -2.279950e+00 1.6738073 NA  -1.362  0.8744
CTRL,FMFF - CTRL,WMFF   -9.809735e-01 0.6770224 NA  -1.449  0.8342
CTRL,FMFF - LOWpH,WMFF  -2.790957e+00 1.6537652 NA  -1.688  0.6954
CTRL,FMFF - CTRL,WMWF   -2.248243e-05 0.8165311 NA   0.000  1.0000
CTRL,FMFF - LOWpH,WMWF  -2.434217e+00 1.6666742 NA  -1.461  0.8284
LOWpH,FMFF - CTRL,FMWF   2.072202e+00 1.5822427 NA   1.310  0.8956
LOWpH,FMFF - LOWpH,FMWF  7.732732e-01 0.4935656 NA   1.567  0.7704
LOWpH,FMFF - CTRL,WMFF   2.072250e+00 1.5822445 NA   1.310  0.8956
LOWpH,FMFF - LOWpH,WMFF  2.622658e-01 0.4206134 NA   0.624  0.9986
LOWpH,FMFF - CTRL,WMWF   3.053201e+00 1.6467732 NA   1.854  0.5828
LOWpH,FMFF - LOWpH,WMWF  6.190059e-01 0.4688054 NA   1.320  0.8914
CTRL,FMWF - LOWpH,FMWF  -1.298929e+00 1.6103573 NA  -0.807  0.9928
CTRL,FMWF - CTRL,WMFF    4.754102e-05 0.4999815 NA   0.000  1.0000
CTRL,FMWF - LOWpH,WMFF  -1.809936e+00 1.5895153 NA  -1.139  0.9483
CTRL,FMWF - CTRL,WMWF    9.809985e-01 0.6770119 NA   1.449  0.8342
CTRL,FMWF - LOWpH,WMWF  -1.453196e+00 1.6029417 NA  -0.907  0.9855
LOWpH,FMWF - CTRL,WMFF   1.298977e+00 1.6103590 NA   0.807  0.9928
LOWpH,FMWF - LOWpH,WMFF -5.110073e-01 0.5164052 NA  -0.990  0.9760
LOWpH,FMWF - CTRL,WMWF   2.279928e+00 1.6738042 NA   1.362  0.8744
LOWpH,FMWF - LOWpH,WMWF -1.542672e-01 0.5563606 NA  -0.277  1.0000
CTRL,WMFF - LOWpH,WMFF  -1.809984e+00 1.5895171 NA  -1.139  0.9483
CTRL,WMFF - CTRL,WMWF    9.809510e-01 0.6770162 NA   1.449  0.8343
CTRL,WMFF - LOWpH,WMWF  -1.453244e+00 1.6029435 NA  -0.907  0.9855
LOWpH,WMFF - CTRL,WMWF   2.790935e+00 1.6537621 NA   1.688  0.6954
LOWpH,WMFF - LOWpH,WMWF  3.567401e-01 0.4927939 NA   0.724  0.9963
CTRL,WMWF - LOWpH,WMWF  -2.434195e+00 1.6666710 NA  -1.461  0.8284

Results are given on the log (not the response) scale.
P value adjustment: tukey method for comparing a family of 8 estimates
Tests are performed on the log scale


Now I don't find the significant difference identified by the GLMM.

Why the GLMM indicates a significant difference and lsmeans not, and why do I get NAs for my df in lsmeans?

The main reason is that you appear to think that the regression coefficients estimate differences from the control condition, when in fact they are more complex than that. Each cell mean is estimated by a linear combination of as many as four regression coefficients, and you need to solve backward to find the meaning of each coefficient. In particular, the coefficient for TrLOWpH:CrossFMWF is in fact an estimate of $(\mu_{22}-\mu_{12})-(\mu_{21}-\mu_{11})$, where $\mu_{ij}$ denotes the mean of the $i$th treatment and $j$th cross. You can verify this from the LSmeans table: $(.6141+.6848)-(1.3874+1.6658)=-1.7543$, which equals the regression coefficient shown.

If you want to understand the relation between the LS means and regression coefficients, you can do

lsmeans(glmm.0, ~ Tr*Cross) @ linfct


and it will display the matrix of linear functions of the regression coefficients.

A secondary issue is that you asked for a Tukey correction on the comparisons, which makes a correction for simultaneously testing 28 contrasts. So even if your interpretation had been correct, the $P$~value would be much higher than the uncorrected one in the table of regression coefficients.

The reason that the d.f. are displayed as NA is simply lsmeans's way of noting that the tests and confidence intervals are asymptotic, based on $z$ statistics rather than $t$ statistics.

• Thanks for your detailed answer, this makes total sense. What would you recommend to test whether there are significant differences between my treatments and crosses? – Mud Warrior May 30 '16 at 13:06
• Well, if you want to compare all 8 combinations with one another, you already did it, and with the right multiplicity correction. If you want a more restrictive subset of comparisons, use by, e.g., lsm = lsmeans(glmm.0, ~Tr*Cross); pairs(lsm, by = "Tr"). See the help pages for these functions. – rvl May 30 '16 at 13:53