I ran a survey containing a multiple choice question (i.e. you can only select 1 of 5 options; the options are considered nominal variables). This survey was completed by 2 different groups (no repeats). There were 1600 responses from the first group, and 600 responses from the second group.

I would like to show that there is no significant difference in how the two groups answered the multiple choice question. For instance, the number of people in Group 1 who selected options 1, 2, 3, 4 and 5 was not significantly different from Group 2.

What statistical test should I use?



You have two alternatives if you are looking at a difference in proportion of people rather than number. Proportion makes more sense in your case.

  1. Using 5 two sample tests of proportions; given your high sample size, it seems Ok to assume normality of the underlying distribution. Null hypothesis is both the groups have the same proportion.
  2. Go for a MANOVA test on all the 5 proportions together.
  • 1
    $\begingroup$ For the 5 tests of proportions, would adjusting alpha to reduce Type I error apply (e.g. something like Bonferroni's correction)? $\endgroup$ – Jane Wayne Nov 30 '17 at 22:09
  • $\begingroup$ For the MANOVA, I think he'd have to use one-way MANOVA with the independent variable being the 2 groups, and dependent variables being the 5 choices, right? If he does detect a statistically significant difference, what would be the follow-up procedures (post-hoc testing)? I've done post-hoc testing for 1-way ANOVA using Tukey's HSD, and I'm wondering in the case of p < alpha, what would be the post-hoc testing procedures for 1-way MANOVA. $\endgroup$ – Jane Wayne Nov 30 '17 at 22:12
  • $\begingroup$ @JaneWayne: You are right about the Bonferroni's correction. For the Post-hoc testing in ANOVA, I use post-hoc testing to find possible pairs that differ significantly when there are more than 2 groups. In this case, I don't think that is required as there are only two groups. $\endgroup$ – honeybadger Dec 1 '17 at 0:23
  • $\begingroup$ Thanks for the replies, I was wondering if it were still possible to use a one-way MANOVA if the dependent variable (the 5 choices) were not continuous (i.e. a nominal variable). $\endgroup$ – Dodo_13 Dec 1 '17 at 16:14
  • $\begingroup$ @Dodo_13 Here the variable under consideration is the proportion of people who opt for a particular choice. This is a continuous variable. If you feel that answer is satisfactory, please accept it $\endgroup$ – honeybadger Dec 1 '17 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.