# Why are p-values in clinical trials often based off of LSmeans

Very often I see clinical trials quoting p-values based upon the differences in treatment effects using the LSmeans. To improve my understanding of this I attempted to learn how to calculate LSmeans using the excellent tutorials written by Russ Lenth.

One thing that has struck me though is that (based upon a toy example using a standard linear regression model) the test statistic and p-value for testing that difference in treatment LSmeans is 0, is identical to the test statistic and p-value for testing that the model $\beta$ coefficient for treatment is equal to 0.

Admittedly this isn't true in the case where you have interaction terms with treatment and I have only considered standard linear regression models so far.

My questions though then are

• Is my assumption that these two quantities are the same correct ?
• If so why do clinical trial results prefer to quote the difference in LSmeans (and confuse readers with the talk of model adjusted means as well as making a significant number of unnecessary calculations) as opposed to just quoting the model treatment effect ?

For reference here is a toy example using R to try and ilistrate the point.

dat <- data.frame(
trt = factor(c(1, 1, 1, 1,   2,2,2,2,2,2)),
eff = c(10, 12, 11, 15,   17,16,11,17,15,18),
sex = c("m", "m","f","m",  "f","f","f","f","m","m")
)

mod <- lm( data = dat , eff ~ sex + trt )
summary(mod)
# Coefficients:
#            Estimate Std. Error t value Pr(>|t|)
# (Intercept)   11.040      1.765   6.256 0.000422 ***
# sexm           1.280      1.695   0.755 0.474898
# trt2           4.200      1.730   2.427 0.045609 *

### Use emmeans library to calculate LSmeans estimate for each TRT
library(emmeans)
x <- emmeans(mod , "trt")
x
# trt emmean       SE df  lower.CL upper.CL
# 1    11.68 1.294913  7  8.618017 14.74198
# 2    15.88 1.038240  7 13.424952 18.33505

### Use pairs function to calulate difference and p-value between TRT LSmeans
# contrast estimate     SE df t.ratio p.value
# 1 - 2        -4.2 1.7304  7  -2.427  0.0456

There are a few simple reasons to report LS means or EM means and their related statistics.

In cases of unbalanced designs, the EM means themselves may not equal the arithmetic means, and EM means may be more meaningful. With your toy data, you can compare,

dat <- data.frame(
trt = factor(c(1, 1, 1, 1,   2,2,2,2,2,2)),
eff = c(10, 12, 11, 15,   17,16,11,17,15,18),
sex = c("m", "m","f","m",  "f","f","f","f","m","m"))

mod <- lm( data = dat , eff ~ sex + trt)

emmeans(mod, ~sex)

with

mean(dat$eff[dat$sex=="f"])

mean(dat$eff[dat$sex=="m"])

If you have more than two levels for a factor, the ability to compare among levels of each factor becomes useful. For example, if we add another sex to the data,

dat2 <- data.frame(
trt = factor(c(1, 1, 1, 1,   2,2,2,2,2,2)),
eff = c(10, 12, 11, 15,   17,16,11,17,15,18),
sex = c("m", "z","f","m",  "f","f","z","z","m","m"))

mod2 <- lm( data = dat2 , eff ~ sex + trt)

z = emmeans(mod2, ~ sex)

pairs(z)

As the model becomes more complex, it becomes more important to have reliable ways to compare among the levels of interactions and to report effects adjusted for other effects in the model.

joint_tests(mod)

may be a useful replacement for Anova or anova in some cases.

In a simple one-factor setting, if the factor has more than two levels, then the default way this is handled by R is to set the first factor level as the reference level. Accordingly, the intercept is an estimate of the mean at the first factor level and the remaining regression coefficients estimate the difference between means at the second, third, ... levels with the first one. In that situation, you are left without estimates of any comparisons that don't involve the first level. Using comparisons based on EMMs, we are able to easily estimate all of the comparisons. For instance, if the factor has 5 levels, there are $5*4/2=10$ pairwise comparisons, and the regression coefficients provide only the four comparisons with the first level. There is also the matter that the EMMs themselves are worth looking at, even before you start comparing them.

Many experiments involve more than one factor, and perhaps covariates as well. Some factors may interact, others may not. You may have factors that interact with covariates. You may have a response transformation or a link function. These all complicate the interpretation of regression coefficients. The emmeans package attempts to provide estimation tools to make it easier to navigate all those possibilities.

• BTW, in that simple one-factor case, the EMMs are indeed just the sample means -- whether or not the data are balanced. But as I say, with covariates or additional factors, the plot thickens considerably. Jul 15, 2018 at 2:41