Choosing between a MANOVA and a series of t-tests when comparing two groups

Question

I have to analyze the data from a study in which two groups of subjects (total sample $n = 30$) were tested on 4 cognitive tasks, all related to executive functions. For each task, I have accuracy score and reaction times. So, the analysis will be made on $4\times 2 = 8$ dependent variables.

In such situation, is it a good practice to conduct a MANOVA with all the DVs (or, in alternative, two separate MANOVAs, one for accuracy scores and one for reaction times), or is it better to analyze the data by a number of t-tests?

Mainly, my doubts arise from the fact that I feel I don't understand when (from a theoretical point of view) a series of dependent variables can be considered a group.

Answer 1

I have recently answered a very similar question, maybe you want to take a look: Assessing group differences on multiple outcomes. However, as the questions have not been marked as duplicates (and I am too new here to attempt it), let me add here the following.

You have a very simple design: only two groups; MANOVA is not a simple procedure and so might be an overkill. Therefore I would start with separate t-tests, and if you can show that your groups differ according to several dependent variables, then perfect. Note that they should better differ consistently, e.g. one of the groups should always have a lower reaction time (and not sometimes lower, sometimes higher, which would be weird). See my linked answer about multiple comparisons. If you don't get convincing and consistent differences with individual t-tests, well, then you can try MANOVA -- again, see my answer with some further tips.

Regarding your theoretical question: I assume you are asking when you can group dependent variables to run a MANOVA. I guess the answer is whenever you want. If you have several dependent variables, then whatever they are you can ask if your groups differ with regard to them.

Answer 2

Many different statistical tests of significance can be applied in research studies. Factors such as the scale of measurement represented by the data, method of participant selection, number of groups being compared and number of independent variables determine which test of significance should be used in a given study. It is important that the researcher select the appropriate test because an incorrect test can lead to an incorrect decision of Type I or Type II error (Gay, Mills & Airasian, 2011).

The t test is used to determine whether two groups of scores are significantly different at a selected probability level. Simple ANOVA is a test of significance used to determine whether scores from two or more groups are significantly different at a selected probability level. Whereas, the t test is appropriate test of difference between the means of two groups at a time (e.g., boys and girls). It is also possible to compute a series of t tests, one for each pair of means. ANOVA is the test for multiple group comparison (Gay, Mills & Airasian, 2011).

MANOVA is the extended form of ANOVA. If a research study uses a factorial design to investigate two or more independent variables and the interactions between them, the appropriate statistical analysis is factorial, or multifactor analysis of variance. This analysis yields a separate F ratio for each independent variable and one for each interaction. When two or more independent variables are analyzed, multivariate analysis of variance MANOVA is used. For example, suppose that we want to consider whether gender and economic status both affect students’ college achievement. MANOVA will allow us to consider both independent variables (gender and economic status) and multiple dependent variables (e.g., college GPA as well as other test scores we may have from maths or language classes). As you can imagine, however, we need a large data set to run increasingly complex analysis with multiple independent and independent variables (Gay, Mills & Airasian, 2011).