can i use a chi-square test to compare the surveys of one sample under two different conditions?

Question

I have a sample of around 50 people, who were placed under two different conditions to perform a task. The outcomes are the time spent on the task and a questionnaire with various points about their subjective experience under the two conditions, where the answers are "disagree, agree, strongly agree etc." Assuming normality,

I'll perform a paired t-test for the time spent on task in the two conditions.

Can i use a chi square to compare the answers to the questionnaire in the two conditions? does the chi square "care" if the samples are paired or independent?

Answer 1

The way you're describing the scale, I am picturing something like this:

      Strong disagree (1) ---- Disagree (2) ---- Agree (3) ---- Strongly agree (4)

If that is a correct interpretation, this is most accurately described as ordinal data, rather than categorical data. The reason for this is that there is an order or directionality in the answers. In the categorical scale of measurement there is no "direction", there are just arbitrary categories. More distinction between the two scales of measurement can be found in pages 3 and 4 here. Furthermore, what you have described is an example of a Likert-type scale, which is usually treated as an ordinal scale.

The $\chi^2$ test can technically be used here, but I wouldn't recommend it because it will treat the answers as entirely separate categories, rather than existing in an order. Instead, we should consider a rank based method that maintains the order of the responses as important.

I would refer to @Glen_b's answer in this question for good starting suggestions on tests for a scenario such as this, where you have 2 dependent (paired) groups and are comparing ranks. Some possible options you have available are:

• Wilcoxon Signed Rank Test (preferred)

• Sign Test

• There are more, but these are two of the more common ones

The Wilcoxon Signed Rank Test will respect the order of your answers while also being appropriate for the paired nature of your comparison groups. Therefore, that would be my suggestion.