multcomp() vs emmeans() for multiple comparisons

Question

I have two sets of models. The first set contains a number of logistic models, fitted using glm(), with different binary dependent variables. The second set contains a number of linear models, fitted with lm(), with different continuous dependent variables.

All models are testing for differences between participants in one of four conditions (three treatments and one control) coded as a factor (with the control as the contrast), while controlling for a number of demographic variables, some of them factors and some of them continuous.

The sample size is roughly 800 per condition (the conditions aren't perfectly balanced), so about 3,200 in total.

In addition to comparing the treatment conditions to the control through summary(), however, I would also like to compare the treatment conditions to one another, to see if some are significantly more effective than others. My question is whether to use multcomp() or emmeans() to do this.

As I understand it (e.g., from here), the main difference is that emmeans() uses a t statistic (assuming that hasn’t changed from lsmeans()) while multcomp() uses a z statistic, and that the latter therefore tends to result in inappropriately small p values and short confidence intervals.

That would seem to recommend emmeans(). Or are there ever other considerations making multcomp() more appropriate?

Answer 1

For glm models, both use a z statistic. In general, there is little difference between using emmeans::contrast() and multcomp::glht() except for user interface. The latter is somewhat harder to use with multi-factor models because there isn't a nice interface for specifying pairwise comparisons of limited groups or marginal averages; but on the other hand, you can specify comparisons in glht() in the same way as in emmeans() by using emm() instead of mcp(). See ? emmeans::emm. Alternatively, you may convert an emmGrid object to summarize in multcomp via the function as.glht().

The default multiplicity adjustment in glht() is the single-step method, and that same method is available as adjust = "mvt" in contrast(). One situation where you may prefer glht() is if you want to use one of the multi-step adjustment methods it offers, which are not available in contrast(). (The exception is that those in p.adjust.methods are available in both.)