# Assessing group differences on multiple outcomes

I have one independent variable with two levels: Gender (Male and Female), and dependent variables will be composed of three test scores: Impulsivity, Assertiveness, and Social Desirability.

My research question is to know the difference between males and females on all three test scores.

I'm confused about the kind of analysis I should use, and whether or not I've actually labelled the dependent and independent variables correctly? Should I be using a MANOVA?

• Hello, Jesse, and welcome to the site! I have to point out that there's no way we on CrossValidated would know whether you've labelled the variables correctly. From a technical point of view, you can label them anything you want - "foo", "bar", ..., and a statistical analysis will work the same way regardless. The rest of your question seems clear enough, though. Jan 19, 2014 at 16:21
• @jbowman: I suppose Jesse was wondering if he correctly classified (labeled) his variables into "dependent" and "independent", i.e. if he correctly understands these terms; this is a valid question (and the answers seems to be positive). Jan 25, 2014 at 14:54

As far as I understand, you primarily want to know if there are any differences between males and females. I agree with @PeterFlom in that you can start with conducting 3 separate two-sample t-tests. Of course with three separate t-tests you are doing multiple comparisons, so be careful in interpreting your p-values. The simplest way to correct for multiple comparisons is to multiply your p-values by the number of comparisons (Bonferroni correction). So if with one of your tests you get uncorrected $p=0.001$, it would correspond to adjusted $p=0.001 \cdot 3=0.003$, which is most probably small enough for you, and then you are done.

On the other hand, if in all three cases you get something like $p=0.4$, then there is not much sense in doing further tests, as there is obviously no difference between groups.

However, if for several (two or three) tests you get results bordering on significance, then you might suspect that combining your independent variables could possibly make the overall difference significant. And this is precisely the question that MANOVA asks, you are right. See my answer here for a figure:

Note that if you do get a significant difference with MANOVA, you would probably face a question of understanding what linear combination of your variables results in a difference, and how to intrepret it. This might turn out to be tricky. See my answer here for some extra thoughts:

In addition, note that in case of one factor (in your case with only two levels), MANOVA is essentially the same thing as LDA, but with an additional procedure of statistical testing. So if I were you, in addition to running MANOVA I would project the data on the first two LDA axes and plot it as a scatter plot. Then you can eyeball the plot and see if two groups (males/females) are visually separated. MANOVA has at least four somewhat different ways of computing p-values, so you will likely get several slightly different estimates of $p$ (Wilks test, Hotelling-Lawley test etc.), and it is always helpful to look at the data with your own eyes to check what is going on.

Update: The previous paragraph does not make sense if you only have two groups; then LDA will only give you one single axis, and not two. Even though I stand by my recommendation in case of more groups, for only two groups you should plot something else.

• Aug 29, 2015 at 20:24
• @ameoba I wonder whether multiple t tests is legitimate if we not only have multiple metrics but also multiple factors each with many factor levels. Two way anova works for for 2 factors, could we just partition by each factor and perform all t tests and just adjust by bonferoni and avoid 2 way anova? Sep 16, 2020 at 9:55

If you just want to know the difference between male and female on these three scores, then you don't need any tests. If you want to assess whether the difference is statistically significant, you could use 3 t tests (if the assumptions are met) or a nonparametric equivalent (if they are not).

You would use MANOVA if you were also interested in accounting for the relationships among the three dependent variables.