# What is correct test I should use to compare multiple response (check all that apply) answers by age group?

I administered a customer service satisfaction survey and one of the questions is a check-all-that-apply asking respondents to check the boxes for the services that they liked most (medical assistance, linkage to medical doctor, housing assistance, therapy, and additional funds). I'd like to test whether there are any differences in counts of responses as a function of age group (55-60, 61-65, 66-70, 71+), i.e. whether certain choices are significantly more likely to be checked by one age group compared to another.

I entered each of the 6 response choices as a separate variable in SPSS, with each answer coded as either 0 (no for no check mark) or 1 (yes for a check mark) for each of the above six response variables.

Would it be appropriate to run six separate 4 (55-60, 61-65, 66-70, 71+) x 2 (yes, no) chi-square tests for independence for each of the six categories? Or, can I just run a test on the counts of the "yes" responses across the six categories by age group as reflected in the following table:

Insight into how to approach this in general and in SPSS in particular will go a long way.

• Hi LeeZee, welcome to CV! On this site there's no need to say "thank you" at the end of your post - it might seem rude at first, but it's part of the philosophy of this site (tour) to "Ask questions, get answers, no distractions" and it means future readers of your question don't need to read through the pleasantries. Apr 20 '18 at 6:25

## 1 Answer

You don't want to run a chi-square test on the large table as it is because it doesn't contain any information on the number of "no" responses. That is, in your table the 61-62 group is always higher in frequency, but I don't know if that's because they had a higher proportion of yes answers, or if it's just because there were more respondents in this group. †

As you suggest, one approach would be to run 6 different chi-square tests of association, one for each "category", with the four age groups on one axis of the table and yes/no on the other axis. This tells you if there are differences in the proportion of responses for each of the age groups, but doesn't allow you to compare across "categories".

Note, however, that the age groups are really ordinal in nature. It may be advantageous to treat age as an ordinal variable. To do this, you would use the Cochran-Armitage test instead of the chi-square test of association. If there is a trend in the data, this approach is often more powerful than a chi-square test of association.

As a different question, you may want to know if one category was more popular than another category, either in the sample overall or within a single age group. This is where you would use Cochran's Q test. You can run a post-hoc analysis to determine which specific categories were more popular than others.

For any of these tests I also recommend examining the size of the differences, either with table or plots of proportions, or with effect-size measurement statistics.

† If this was a "choose one" type of question, then this analysis would work, because the categories would be mutually exclusive.

• Sal, thank you for your thoughts. They're very helpful. Apr 20 '18 at 18:21
• Sal, thank you for your thoughts--very helpful. All 501 respondents of the sample are providing responses to the six categories; So, as you said the 61-65 group just has a higher proportion of yes answers. So, if that frees up chi-square as an option, it still sounds like since it's not a "choose one" type question, the categories are not independent and therefore chi-square is not viable...So, if I want to use Cochran-Armitage test instead after turning age into ordinal scale, is there a way to do in SPSS? Lastly, I will also run Cochran's Q test to see if one category more popular in SPSS. Apr 20 '18 at 18:37
• I don't really understand the question in the first half of your comment. You can use chi-square on the 2 x 4 tables, but not on the 6 x 4 table shown in your question..... If you use C-A, you will have two levels in the nominal variable (yes/no). Because of this, the C-A test is the same as the linear-by-linear test for two ordinal variables. I don't use SPSS, but SPSS outputs this with other statistics of association for tables. I think it's called "linear-by-linear" or something similar in the output. Apr 20 '18 at 18:48
• Sal, sorry about the confusion--Yes, I was just saying that it sounds like I can't do a chi-square on the 6 x 4 table because of the lack of independence of the categories as you pointed out...Thank you, I'll look into the linear-by-linear option for tests of association in SPSS. Apr 20 '18 at 19:24
• Sal, I wonder if you can clarify something for me. So if the same 501 respondents answered "yes" or "no" to all of the categories, then there are the same number of respondents in each age group. So, why then couldn't I do a chi-square on the 6 x 4 frequencies of "yes" responses shown in the table above? You also mentioned that since the question wasn't a "choose one" type of question that the categories aren't mutually exclusive, and in this case the frequencies for each category share respondents with frequencies of other categories--Is this why it doesn't work? Apr 23 '18 at 16:15