For a MANOVA with $n$ variables, I would like to do pairwise comparisons between $k$ levels for one of the variables.

What is the suitable method to adopt for this while adjusting $\alpha$ for the $k(k-1)$ multiple comparisons?

  1. Is multiple Hotelling $T^2$ tests along with Bonferroni correction or FDR/pFDR appropriate? FDR/pFDR q values would be preferable as the $\beta$ value is important here.

  2. Any suggestions for R packages to do the same? (Particularly for MANOVA post-hoc multiple comparisons}

  3. How to test the null hypothesis $H_0^j:|\mu_1^j-\mu_2^j|\ge\delta$ instead of $H_0^j:\mu_1^j=\mu_2^j$ as in an equivalence test for the multiple comparisons?


Based on the answer and further comment by rvl, I was able to explore and come up with the following.

# Use the `oranges` dataset in `lsmeans` package.
# multivariate linear model
oranges.mlm <- lm(cbind(sales1,sales2) ~ price1 + price2 + day + store,
                  data = oranges)
# Get the least square means
oranges.Vlsm <- lsmeans(oranges.mlm, "store")
# Multiple comparisons with fdr p value adjustment
test(contrast(oranges.Vlsm, "pairwise"), side = "=",  adjust = "fdr")
# With threshold spcified
test(contrast(oranges.Vlsm, "pairwise"), side = "=",  adjust = "fdr", delta = 0.25)
  • 1
    $\begingroup$ I think for the equivalence tests you mean $H^{j}_{0}: |\mu^{j}_{1}-\mu_{2}^{j}| \ge \delta$? As in $H^{j}_{01}: \mu^{j}_{1}-\mu_{2}^{j} \ge \delta$ –or– $H^{j}_{02}: \mu^{j}_{1}-\mu_{2}^{j} \le -\delta$ along the lines of two one-sided tests, yes? $\endgroup$
    – Alexis
    Commented Mar 12, 2015 at 19:45
  • $\begingroup$ @Alexis you are right. Have edited the question accordingly. $\endgroup$
    – Crops
    Commented Mar 13, 2015 at 7:34

1 Answer 1


For an R package, you might take a look at lsmeans. For mlm models, it sets up the multivariate response as if it were a factor whose levels are the dimenstions of the response. Then you can do estimates or contrasts of those, with or without other factors being involved. See the example for the MOats dataset that accompanies the package.

It also supports equivalence tests via providing a delta argument in summary or test. A section of the vignette (see vignette("using-lsmeans")) covers equivalence testing.

  • $\begingroup$ I'll add a comment that what lsmeans won't do is a family of multivariate tests, and maybe that's what is at issue... $\endgroup$
    – Russ Lenth
    Commented Mar 12, 2015 at 19:48
  • $\begingroup$ @rvl But there is an option to do pairwise comparison of least square means derived from a multivariate model in lsmeans. See the edit to the question for the code. $\endgroup$
    – Crops
    Commented Mar 13, 2015 at 7:37
  • $\begingroup$ Ok, it seems to provide what you need then. Based on an earlier version of the question, I had thought that you might have wanted to do multiple comparisons of multivariate means, with an equivalence twist added, and that'd be a whole new ballgame. $\endgroup$
    – Russ Lenth
    Commented Mar 13, 2015 at 19:02
  • $\begingroup$ @rvl, Isn't the above case given in the edit same as "multiple comparisons of multivariate means for equivalence", as two dependent variables are specified in the LHS of the linear model (sales1 and sales2)? $\endgroup$
    – Crops
    Commented Mar 16, 2015 at 9:37
  • $\begingroup$ @Crops, I think we are stuck on a semantic issue. The important thing is whether you were able to do the equivalence tests that you needed to do. Is that the case? $\endgroup$
    – Russ Lenth
    Commented Mar 16, 2015 at 12:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.