2
$\begingroup$

Confidence intervals on means in regression widen as predictions approach the edges of model fit. My intuition then leads me to believe that if I do pairwise comparisons using my model, adjusting for a covariate, I expect that confidence intervals on the difference of means (e.g. u1 - u2) should also get wider near the edges. However, when I use lsmeans, this appears to not be the case. The p-values all come out the same. Am I misusing the tool, or is my intuition wrong here? If so, can you help me understand why?

Below is some R code for generating an example:

library(dplyr)
library(lsmeans)

clamp <- function(x, low, high) {
  ifelse(x < low, low, ifelse(x > high, high, x))
}

set.seed(101)

nexp <- 5
ntrt <- 3
data <- data_frame(treatment = factor(rep(LETTERS[1:ntrt], each = nexp)),
                   x = rnorm(length(treatment), sd = 0.06) + 
                     rep(seq(0.1, 0.5, length.out = nexp), ntrt),
                   y = rnorm(length(treatment), sd = 0.03) + 
                     rep(seq(0.5, 0.15, length.out = nexp), ntrt) +
                     rep(seq(0.3, 0.1, length.out = ntrt), each = nexp)) %>%
  mutate_each_(funs(clamp(., low = 0.05, high = 0.95)), 
               vars = c("x", "y"))

mod.lm <- lm(y ~ treatment + x, data = data)
mod.lsm <- lsmeans(mod.lm, ~ treatment | x,
                   at = list(x = c(0.1, 0.3, 0.5)))

pairs(mod.lsm)

This gives the statistical output:

x = 0.1:
 contrast estimate      SE df t.ratio p.value
 A - B     0.05502 0.02355 11   2.336  0.0921
 A - C     0.18409 0.02348 11   7.842  <.0001
 B - C     0.12907 0.02360 11   5.470  0.0005

x = 0.3:
 contrast estimate      SE df t.ratio p.value
 A - B     0.05502 0.02355 11   2.336  0.0921
 A - C     0.18409 0.02348 11   7.842  <.0001
 B - C     0.12907 0.02360 11   5.470  0.0005

x = 0.5:
 contrast estimate      SE df t.ratio p.value
 A - B     0.05502 0.02355 11   2.336  0.0921
 A - C     0.18409 0.02348 11   7.842  <.0001
 B - C     0.12907 0.02360 11   5.470  0.0005

This matches the value given by JMP, which doesn't give the option of specifying different locations on the covariate:

JMP pairwise comparisons

$\endgroup$
2
$\begingroup$

Note that not only do the CIs not get wider, but the estimated differences are identical in all three cases. The reason is because your model is additive. The LS mean for the $i$th treatment at a given $x$ value is $$ \hat y_{i,x} = b_0 + \hat\tau_i + b_1 x $$ where $\hat\tau_i$ are treatment effects, $b_0$ is the intercept, and $b_1$ is the trend for $x$.

If you take the difference of two of these at different $i$s but the same $x$ value, you are just left with $\hat\tau_i - \hat\tau_{i'}$; the $b_0$ and $b_1x$ terms cancel out. So the differences don't depend at all on $x$.

If you include treatment:x in the model, it will be a different story.

(Note: In the expressions above, I am showing a different parameterization than is probably actually used by R; but all parameterizations are equivalent when it comes to describing properties of estimable contrasts.)

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.