1
$\begingroup$

I have a relatively straightforward question but I haven't been able to find the answer. Consider this example:

require(emmeans)
fiber.lm <- lm(strength ~ diameter*machine, data=fiber)

My hypothesis is that for each of the three machines, there is a relationship between strength and diameter, i.e. the slope for each machine is significantly different from 0. To test this, I use the emmeans package as follows:

summary(emtrends(fiber.lm, ~"machine", var = "diameter"), infer = TRUE)

My question is: should I use multiplicity correction on the pvalues and confidence intervals (e.g. by adding adjust="sidak") to conclude whether the slopes are significantly different from 0 or not? If yes, what may be the reason why this not done by default like when using pairwise comparisons in the emmeans package?

I understand there are no fixed rules about this, but I haven't been able to find discussions or examples in the literature about this specifically. In tutorials on the emmeans package, I have only found instances where no adjustment is performed (e.g. https://stats.idre.ucla.edu/r/seminars/interactions-r/#s4b and https://psu-psychology.github.io/r-bootcamp-2018/talks/correlation_regression.html#pairwise-differences-and-simple-slopes-in-regression).

$\endgroup$
2
$\begingroup$

First, a multiplicity correction is a multiplicity correction. That is, the importance of using one or not is due solely to the fact that you are doing more than one test, and the risks of making an error somewhere in that family of tests. Whether it is slopes or pairwise differences or whatever is not of key importance.

Second, there is also the possibility of doing one joint test, via test(emtrends(...), joint = TRUE). That avoids the multiplicity issue. Or rather, it incorporates it, as it considers all slopes at once.

$\endgroup$
2
  • $\begingroup$ Thank you for your answer! Regarding your first comment: yes, that's how I see it as well. That's why I'm confused as to why when doing pairwise comparisons, the default is to correct for multiple testing, and with slopes, it's not the default and not commonly described, e.g. in the references I gave... Regarding the joint test: I have been trying to read up on it, and as I understand it, that would not tell me whether each of the slopes are significantly different from 0 or not, or does it? $\endgroup$ Apr 24 '20 at 11:07
  • $\begingroup$ Right. An omnibus test doesn't tell you which slopes will test nonzero -- or even if any of them will. Only that some linear function of them will test nonzero. $\endgroup$
    – Russ Lenth
    Apr 24 '20 at 17:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.