Post-hoc tests for MANOVA: univariate ANOVAs or discriminant analysis?

Question

I am using a MANOVA test to compare nine different dependent variables (from neuropsychological and neuropsychiatric assessment) between three groups. The output shows a significant influence from GROUP on my variables ($p < .001$).

Ofcourse, I am interested in how the three groups influence every dependent variable. I have studied Field's "Discovering Statistics Using IBM SPSS Statistics" chapter 16, and he states that the preferred post-hoc analysis is a discriminant analysis, because of the linear combination in which the dependent variables are related to group membership in a MANOVA. Discriminant analysis could account for this linear combination, so Field states.

Otherwise, I read some literature, on basic statistical sites, where is stated that I can use multiple univariate ANOVA's with Bonferroni correction, and use a post-hoc on these univariate ANOVA's when they are significant.

Which of these method's is better? That is, which will make my chance of a Type I or II error the least?

Answer 1

"Significant influence from group" means that $H_0: {\mu_1}=\mu_2=\mu_3$ has been rejected, where $\mu_i$ is the mean vector of the dependent variables in group $i$. This can happen if $\mu_1=\mu_2\neq \mu_3$. In this case, discriminant analysis between group 1 and 2 would fail. You would have first to decompose the overall hypothesis into $\mu_1 = \mu_2$, $\mu_2 = \mu_3$ and $\mu_1 = \mu_3$. There, of course, multiplicity adjustments (e.g. Bonferroni) are again necessary.

Even if it does not fail, discriminant analysis gives you rather estimates of the effects, not test results. If you are in fact interested in such a tool (e.g. in order to diagnose to which group a new patient would belong), discriminant analysis will of course still be necessary.

Multiplicity adjustments of the hypotheses $H_0 ^ {j}:\;\mu_1^{j}=\mu_2^{j}=\mu_3^{j}$, where $j$ denotes a dependent variable, can be done with Bonferroni-method. The interpretation of a significant result would be that in the dependent variable you identified not all groups have equal means. Usually you would want to decompose this result as well into pairwise comparisons as above. Also you have to keep in mind that it may happen that you can reject the global hypothesis but fail with the post-hoc analyses.

Your last question: As Bonferroni is quite conservative, you may consider using different methods, e.g. like in the SimComp R-package. This would estimate the unknown dependency between the variables. Said information would lead to a less conservative adjustment, thus, better power.

Answer 2

These two follow-up approaches have very different goals!

• Univariate ANOVAs (as follow-ups to MANOVA) aim at checking which individual variables (as opposed to all variables together) differ between groups.

• Linear Discriminant Analysis, LDA, (as a follow-up to MANOVA) aims at checking which linear combination of individual variables leads to maximal group separability and at interpreting this linear combination.

  This question asked about one-way MANOVA with only a single factor, but see here for the [more complicated] case of factorial MANOVA: How to follow up a factorial MANOVA with discriminant analysis?

So e.g. if your individual variables are weight and height, then with univariate ANOVAs you can test if weight and height, separately, differ between groups. With LDA you can find out that the best group separability is given by, say, 2*weight+3*height. Then you can try to interpret this linear combination.

So the choice between these two follow-up approaches entirely depends on what you want to test.

---

Two further remarks

First, if you are "interested in how the three groups influence every dependent variable" (i.e. individual DVs are of primary interest), then you should arguably not run MANOVA at all, but go straight to univariate ANOVAs! Correct for multiple comparisons (note that Bonferroni is very conservative, you might prefer to control false discover rate instead; but see comment below for another opinion), but proceed with univariate tests. After all, nine DVs are not a lot. If, instead, you are interested in whether groups differed at all (and maybe in what respect they differed the most) but do not care so much about individual DVs, then you should use MANOVA. It all depends on your research hypothesis.

Second, it sounds though as if you might have no pre-specified hypothesis about which DVs should be influenced by group, and what exactly this influence should be. Instead, you probably have a bunch of data that you wish to explore. It is a valid wish, but it means that you are doing exploratory analysis. And in this case my best advice would be: plot the data and look at it!

You can plot a number of things. I would plot distributions of each DV for each of the three groups (i.e. nine plots; can be density plots or box plots). I would also run linear discriminant analysis (which is intimately related to MANOVA, see e.g. my answer here), project the data onto the first two discriminant axes and plot all your data as a scatter plot with different groups marked in different colours. You can project original DVs onto the same scatter plot, obtaining a biplot (here is a nice example done with PCA, but one can make a similar one with LDA too).