Tukey correction in emmeans

Question

I have a question about the Tukey correction in emmeans. See the example below. It says "P value adjustment: tukey method for comparing a family of 3 estimates." Does this mean that the error rate for all of these comparisons (which were run together) is 5%? What does "a family of 3 estimates" refer to?

The two factors here were "group," which had 3 levels, and "condition," which had 4 levels, so there are a total of 12 comparisons. If each comparison had been run separately, the error rate would have been 5% comparison. I'm hoping that because these were all run together with the Tukey correction, the error rate is down to 5% for all of them. What I'm confused about is the reference to "3 estimates," and I'm wondering if this means the error rate is 5% for each set of three group comparisons (one set per condition), meaning the error rate is actually 1 - 0.95^4 (about 18%)?

Thank you!

> emmeans(model_1, pairwise ~ group | condition)
$emmeans
condition = EN-TVJT-R-BRF:
 group emmean     SE  df lower.CL upper.CL
 EN    0.0456 0.0174 219  0.01129   0.0799
 HS    0.1515 0.0211 368  0.11004   0.1930
 SB    0.0208 0.0246 500 -0.02745   0.0691
 
condition = EN-TVJT-R-BRT:
 group emmean     SE  df lower.CL upper.CL
 EN    0.9848 0.0174 219  0.95043   1.0191
 HS    0.8712 0.0211 368  0.82974   0.9127
 SB    0.9479 0.0246 500  0.89964   0.9962
 
condition = EN-TVJT-R-THF:
 group emmean     SE  df lower.CL upper.CL
 EN    0.0254 0.0174 219 -0.00891   0.0597
 HS    0.1742 0.0211 368  0.13277   0.2157
 SB    0.0417 0.0246 500 -0.00661   0.0899
 
condition = EN-TVJT-R-THT:
 group emmean     SE  df lower.CL upper.CL
 EN    0.9898 0.0174 219  0.95548   1.0241
 HS    0.8939 0.0211 368  0.85247   0.9354
 SB    0.9688 0.0246 500  0.92047   1.0170
 
Degrees-of-freedom method: kenward-roger 
Confidence level used: 0.95 

$contrasts
condition = EN-TVJT-R-BRF:
 contrast estimate     SE  df t.ratio p.value
 EN - HS   -0.1059 0.0266 783 -3.983  0.0002 
 EN - SB    0.0248 0.0294 783  0.842  0.6768 
 HS - SB    0.1307 0.0317 783  4.117  0.0001 
 
condition = EN-TVJT-R-BRT:
 contrast estimate     SE  df t.ratio p.value
 EN - HS    0.1136 0.0266 783  4.271  0.0001 
 EN - SB    0.0368 0.0294 783  1.252  0.4230 
 HS - SB   -0.0767 0.0317 783 -2.417  0.0420 
 
condition = EN-TVJT-R-THF:
 contrast estimate     SE  df t.ratio p.value
 EN - HS   -0.1488 0.0266 783 -5.597  <.0001 
 EN - SB   -0.0162 0.0294 783 -0.552  0.8454 
 HS - SB    0.1326 0.0317 783  4.177  0.0001 
 
condition = EN-TVJT-R-THT:
 contrast estimate     SE  df t.ratio p.value
 EN - HS    0.0959 0.0266 783  3.606  0.0010 
 EN - SB    0.0211 0.0294 783  0.716  0.7543 
 HS - SB   -0.0748 0.0317 783 -2.357  0.0489
 
Degrees-of-freedom method: kenward-roger 
P value adjustment: tukey method for comparing a family of 3 estimates

Answer 1

This is documented in ? summary.emmGrid. The Tukey correction is applied to each set of comparisons of three means. If you want all 12 comparisons to be adjusted as one family, you need to do something like

EMM <- emmeans(...) # as shown
test(EMM$contrasts, by = NULL, adjust = "mvt")

This combines all of them into one family and applies the multivariate t adjustment. The Tukey adjustment can be used only with a single family of pairwise comparisons and won't be allowed.