Is is statistically correct to claim that I proved (with p-value < 0.05) that "mean of versicolor group > mean of setosa group" if all I did was to bootstrap a 95% confidence interval for "setosa - versicolor" contrast like in this code?


m <- lm(Sepal.Length ~ Species, data=iris)
m.boot <- Boot(m, function(.) {
  t <- summary(emmeans(., pairwise ~ Species))$contrasts
  setNames(t$estimate, t$contrast)[1]
## Bootstrap quantiles, type =  bca 
##                        2.5 %     97.5 %
## setosa - versicolor -1.104946 -0.7687479

If "Yes", what would be the proper reference to support such analysis?

I suspect there must be a catch here. I mean, why would anyone bother to develop packages like "ARTool" if a simple bootstrap will do... If so, can you suggest an alternative way? Preferably, unlike ARTool, one allowing to fit arbitrary linear models and visualize confounder-controlled group means on the original scale (not transformed to ranks).

In reality we need to analyze several contrasts in our (not quite balanced and normally distributed) observational experiment, designed as 3-way ANCOVA with a confounding variable.

  • $\begingroup$ Is it possible to pare down this post into a single, focused question with a minimally reproducible example? I am having a hard time understanding what you're really asking. $\endgroup$ – AdamO Feb 7 '18 at 15:33
  • $\begingroup$ to AdamO: Please tell me which parts are not reproducible, I will be happy to clarify. I thought linear model with 2 factors is "minimal" to fully illustrate my questions: regression is complicated enough, so that post-hoc analysis is "a must" to interpret it. My (oversimplified) main question: Is it correct to "directly bootstrap" just a single contrast (say "A,X - B,X") to get 95% CI and p-value for it? $\endgroup$ – Daniil Feb 7 '18 at 16:47
  • $\begingroup$ minimally reproducible means that the fewest lines of code necessary to convey the problem are presented. Perhaps you can focus on another example with a single contrast. $\endgroup$ – AdamO Feb 7 '18 at 17:05
  • $\begingroup$ Regarding “forcing emmeans...,” you might consider using emmeans::contrast with a specified list of contrast coefficients, and/or a by variable. It’s all right there in the documentation. $\endgroup$ – rvl Feb 8 '18 at 1:10
  • $\begingroup$ to rvl: Thank you, of course I RTFM about "list of contrast coefficients". I could unite the coefficient lists from "pairwise ~ abc|xy" and "pairwise ~ xy|abc". But do you agree that this is an extremely inconvenient way of specifying "custom" sets of contrasts as compared to using human readable formula? $\endgroup$ – Daniil Feb 8 '18 at 9:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.