# Why are the confidence intervals so large for the difference of differences?

I have run a generalized linear mixed effects model with the glmmTMB package to determine if there is an interaction between two categorical predictors, treatment and location, in predicting the number of birds detected.

mod3 <- glmmTMB(sum.50 ~ location*trtmt + julian2 + min_aft_sunrise2 + mintemp2 + (1|week) + (1|site:day), data = wbnu_sub, ziformula=~1, family=poisson)


I'm interested in looking at comparing the levels of treatment and location to each other, so I have used the emmeans package to computer the difference of differences.

mod3.emm <- emmeans(mod3, ~ location|trtmt, adjust = "none")
locdiffs <- contrast(mod3.emm, "revpairwise", adjust = "mvt")
locdiffs

trtmt = pulsed_nofood:
contrast   estimate    SE  df t.ratio p.value
near - far   -0.273 0.586 124 -0.467  0.6416

trtmt = constant:
contrast   estimate    SE  df t.ratio p.value
near - far    2.024 0.376 124  5.381  <.0001

trtmt = pulsed_food:
contrast   estimate    SE  df t.ratio p.value
near - far    2.737 0.728 124  3.758  0.0003

Results are given on the log (not the response) scale.

c <- confint(contrast(locdiffs, "revpairwise", by = NULL, adjust = "mvt"))
c

 contrast                                          estimate    SE  df lower.CL upper.CL
near - far,constant - near - far,pulsed_nofood       2.297 0.696 124    0.652     3.94
near - far,pulsed_food - near - far,pulsed_nofood    3.011 0.935 124    0.801     5.22
near - far,pulsed_food - near - far,constant         0.714 0.820 124   -1.223     2.65

Results are given on the log (not the response) scale.
Confidence level used: 0.95
Conf-level adjustment: mvt method for 3 estimates


These results seemed okay, except for when I back-transformed these I realized that the confidence intervals are gigantic!

c.response <- confint(contrast(locdiffs, "revpairwise", by = NULL, adjust = "mvt", type = "response"))

contrast                                          ratio    SE  df lower.CL upper.CL
near / far,constant / near / far,pulsed_nofood     9.95  6.93 124    1.919     51.6
near / far,pulsed_food / near / far,pulsed_nofood 20.31 18.99 124    2.228    185.1
near / far,pulsed_food / near / far,constant       2.04  1.67 124    0.294     14.2

Confidence level used: 0.95
Conf-level adjustment: mvt method for 3 estimates
Intervals are back-transformed from the log scale


And the same is for the locdiffs:

locdiffs.response <- contrast(mod3.emm, "revpairwise", adjust = "mvt", type = "response")

trtmt = pulsed_nofood:
contrast    ratio     SE  df lower.CL upper.CL
near / far  0.761  0.446 124    0.238     2.43

trtmt = constant:
contrast    ratio     SE  df lower.CL upper.CL
near / far  7.566  2.845 124    3.594    15.93

trtmt = pulsed_food:
contrast    ratio     SE  df lower.CL upper.CL
near / far 15.447 11.252 124    3.653    65.32

Confidence level used: 0.95
Intervals are back-transformed from the log scale


Why is that? Does it have to do something with the log of differences turns into a ratio? If so, why does that mean the standard errors become gigantic?

• This is a fundamental property of logarithms. But in light of your stated objective--"determine if there is an interaction between two categorical predictors"--does it matter how wide the confidence intervals are? – whuber Jun 18 at 14:31
• @whuber, I'm having trouble understanding it since I never counted more than 5 birds at a site. Not sure how to explain it to someone in that context. – Rachael Jun 18 at 14:49
• Note that if you compute exp(.65) and exp(3.94) you get about 1.9 and 51 — consistent with intervals after back-transforming. So this looks right in transforming log odds ratios to odds ratios. Understand that odds are defined as p/(1-p) and you are taking the ratio of two such quantities with only 5 observations, you can expect CIs to be very wide because you can’t accurately estimate anything with very little data. – rvl Jun 18 at 16:41
• When you say odds are defined as p/(1-p) what is p from? Also, I had more than 5 observations, I had 11-12 for each treatment (and each treatment is replicated across sites). – Rachael Jun 19 at 2:17