0
$\begingroup$

I have the following analysis: I ran a MANOVA with two DVs (both empathy scales) and an IV (gender). Now the $p$-value (.03) tells me that there is a significant difference between men and women. However, I do not know which is higher on which variable. When I then look at the ANOVA table (given with SPSS) output, it tells me that there is no significant difference (all $p$-values larger than .05). How do I interpret this? Had men and women score differently or not?

Also, my supervisor asked me to look at the means, however they are only different by maybe one point, so I cannot see there which one should be larger (and the MANOVA only tells me the joint effect of the two DVs together).

When I ran a t-test to have a look at the individual means, they were also non-significant. I am a bit lost what to do with this and would highly appreciate some help in regards how to put this interpretation into words.

$\endgroup$
  • 1
    $\begingroup$ What have you looked at? I searched the obvious on Google and the first link seemed pretty straightforward (it had errors but at least it would put you on the right track). Two things could improve your question. The first is to provide clarification on what all your p-values come from specifically (i.e. what test is 0.03?); and the second is to explain what you have looked at and ask for clarification where you were stuck. $\endgroup$ – John Jul 24 '13 at 17:01
1
$\begingroup$

This answer does not completely address your question about MANOVA and the distinction between univariate and multivariate tests but note that p-values are not all there is to statistical analysis. Looking at the means/plotting them is in any case a good idea.

Thus, it's perfectly possible for a difference to be significantly different from 0 and still very small in some sense. I don't know if “one point” is a lot on your empathy scales but what the test tells you is merely that it would be unlikely to observe it if the rating from men and women were exactly the same. Even if all the tests you performed would reject the null hypothesis, you would still need something else (plots, descriptive statistics, standardized or unstandardized indices of effect size, confidence intervals around these effect sizes, etc.) to fully understand the magnitude of the effect because it's simply not what the test itself is about.

Furthermore, one important point is that “the difference between significant and non-significant is not itself significant”. The logic of hypothesis testing requires you to select an error level (often 5%) and stick to it but there is nothing magical about this threshold. Informally, p = .03 is not very strong evidence of a difference and p = .06 is not evidence that the difference is exactly 0. This point is particularly important when comparing different studies or different groups but there is nothing really surprising about two tests producing slightly higher or lower p-values even with the same data and no need to read too much into it.

$\endgroup$
1
$\begingroup$

The problem you find concerning a statistically significant MANOVA output, but no statistically significant univariate ANOVA's, lies in the very nature of a MANOVA.

A MANOVA differs from several univariate ANOVA's in the fact that you control for mutual relationships. So, you do not only test several 't-tests' or ANOVA's, but there is more to it. This can explain the fact that you do not find seperate significant results, although you're MANOVA is statistically significant. So, apparently gender has an effect on the combination of empathy scales, but not on the empathy scales independently.

For an excellent paper on this difference, check Huberty and Morris' article on the difference between an ANOVA and MANOVA.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.