# Over enthusiastic MANOVA in R

In recent research I pursue a MANOVA model on a dataset comprised of 50 measured characters and several grouping factors that I need to test. When I use something like this

summary(manova(malesM ~ popMales*manageMales*biomeMales), test = "Wilks")


I get 2.2e-16 significance for every possible output (also with other tests beside Wilks). The same thing I noticed when I use dummy, randomly generated factors. My data comes from a real-life study and I really can not belive that almost any theoretical, grouping factor will be significant. Any advice on how to refine my model, and to test this further?

-
How many rows of data are we talking about here? –  Roman Luštrik Dec 6 '11 at 12:26
853 rows to be exact –  Ian Stuart Dec 7 '11 at 9:24

This suggests to me that you may have one or a few observations that are extreme multivariate outliers for your dependent variables malesM, and that the MANOVA test is just picking up that the mean of any group containing such an outlier is far from the mean of those that don't.

To check for this, you could look at the Mahalanobis distances of all the observations from the sample mean using the sample covariance matrix.

-
Nice advice @onestop but unfortunately all Mahalanobis distances are quite similar for all observations in question. –  Ian Stuart Dec 7 '11 at 9:23
Oh well. In retrospect, maybe that's close to inevitable due to the effect of the curse of dimensionality on distance functions. –  onestop Dec 7 '11 at 15:18
Wow I was not aware that this thing can be so potent but it seems so. What can I do? –  Ian Stuart Dec 13 '11 at 10:03
50 measured characters may be too many $Y$ variables with a sample size of 853. The various tests all assume the $Y$ variables have a multivariate normal distribution and even then the $F$ tests are based on asymptotic approximations (for more than 2 $Y$ variables and more than 2 model d.f.). Though (at least some of) the tests are reasonably robust to departures from multivariate normality with reasonable sample sizes and a handful of $Y$ variables, I suspect that for many $Y$ variables the tests may become highly sensitive to departures from multivariate normality and/or the asymptotic approximations start to require really huge sample sizes.