How do I conduct a $\chi^2$ test on two different surveys?

Question

I conducted a pre and post survey that consisted of five questions with answers being either agree or disagree. The survey was to assess if a persons perception of Pit Bulls was improved after they interacted with it.

The first or pre survey was before the individual interacted with the dog. After they did, the respondents immediately were given the same survey (the post survey).

I now have the results of the two surveys and am uncertain how I am to conduct the $\chi^2$ test to see if there was a significant improvement in the perception of the dog?

Specifically I do not know if I am to assess each question individually or if the numbers are compiled for all questions.

Answer 1

I think you want to sum the questions, probably. If you have only 5 questions and each is yes/no, then you should figure out which answer relates to favorable impression of the dog, sum those for each person (so each person has 2 scores from 0 to 5, one before and one after). Then you want to do some plotting and descriptive statistics to see what the scores are like. Then you can choose a statistical test. A test of medians might be appropriate, as might a permutation test.

Answer 2

If your data are on an ordinal scale I might evaluate individual questions in terms of paired differences with probably a Wilcoxon signed rank test. But I would have to adjust for multiplicity if I test every question. If you want to compare the distributions of answers before and after there are two sample goodness of fit tests such as Kolmogorov-Smirnov that can be used. Again if you do this by individual questions you need to address the multiplicity issue. I don't know about your survey but many valid surveys categorize their questions into domains which might be a small number (3-5) whereas you might have 25-50 or more individual questions. The survey is designed so that the scores can be summed. Then you compare the summed scores using Wilcoxon signed rank or the Kologorov - Smirnov comparison. Multiplicity adjustment is then more reasonable.