# compare contrasts from different models with emmeans

I'm trying to compare two contrasts from different models (each model from data from different individuals), and I'm wondering if that is possible using the estimate, SE and t.ratio information from each contrast.

Just as an example, let's pretend I have two independent models mod1 and mod2:

data1 <- mtcars[1:2]
data$$cyl <- factor(data$$cyl)
rownames(data) <- paste("id",1:nrow(data), sep="")
mod1 <- lm(mpg ~ cyl, data=data)

data2 <- mtcars[1:2]
data2$$cyl <- sample(data$$cyl)
names(data2)[names(data2) == 'cyl'] <- 'cyl2'
rownames(data2) <- paste("id",1:nrow(data2)+nrow(data2), sep="")
data2$$mpg = runif(length(data$$mpg))
names(data2)[names(data2) == 'mpg'] <- 'mpgrand'
mod2 <- lm(mpgrand ~ cyl2, data=data2)


Using emmeans to compute the contrasts:

em1 <- emmeans(mod1, pairwise~cyl)
em2 <- emmeans(mod2, pairwise~cyl2)

em1$contrasts contrast estimate SE df t.ratio p.value cyl4 - cyl6 6.92 1.56 29 4.441 0.0003 cyl4 - cyl8 11.56 1.30 29 8.905 <.0001 cyl6 - cyl8 4.64 1.49 29 3.112 0.0112 em2$contrasts
contrast    estimate    SE df t.ratio p.value
cyl24 - cyl26  -0.0371 0.129 29  -0.288  0.9554
cyl24 - cyl28  -0.0233 0.107 29  -0.218  0.9743
cyl26 - cyl28   0.0137 0.123 29   0.111  0.9932


Since I know the estimate, SE, df and t.ratio values of each contrast, can I now somehow compare em1's first contrast

contrast    estimate   SE df t.ratio p.value
cyl4 - cyl6     6.92 1.56 29   4.441  0.0003


with em2's first contrast

contrast    estimate    SE df t.ratio p.value
cyl24 - cyl26  -0.0371 0.129 29  -0.288  0.9554



in order to claim that the em1 contrast value is significantly greater than the em2 contrast value?

I am not quite sure what your underlying question/hypothesis is but given from what you said, why not calculating the difference between mpg and mpgrand and then run a new model on mpg_diff for example?

data$$mpg_diff <- data$$mpgrand - data$$mpg mod3 <- lm(mpg_diff ~ cyl, data = data) em3 <- emmeans(mod3, pairwise~cyl) > em3$$emmeans
cyl emmean    SE df lower.CL upper.CL
4    -26.3 1.009 29    -28.4    -24.2
6    -19.2 1.264 29    -21.8    -16.6
8    -14.6 0.894 29    -16.4    -12.8

Confidence level used: 0.95

\$contrasts
contrast    estimate   SE df t.ratio p.value
cyl4 - cyl6     -7.1 1.62 29  -4.392  0.0004
cyl4 - cyl8    -11.7 1.35 29  -8.684  <.0001
cyl6 - cyl8     -4.6 1.55 29  -2.972  0.0158

P value adjustment: tukey method for comparing a family of 3 estimates


I don't think it makes much sense to compare the contrasts of different models (i.e. fit with different dependent variables) and then run some sort of significance test on those contrasts.

• Thanks for your answer Stefan! Sorry, I should have been clearer in my question. The two models actually come from data from different individuals, so calculating the difference does not make sense in this case. I have updated the question to reflect that point. Commented Mar 12 at 7:43