I am trying to determine the most appropriate test for parts of my dataset. I have been using a chi2$\chi^2$ test, but I am now realising it might not be the most appropriate. I am using Stata.
I have a survey consisting of five-point Likert scales, i.e. Definitely not, Probably not, Unsure, Probably, Definitely. I have 233 respondents answering many different questions.
As an example, these questions ask: Do you think X should be available? Do you think Y should be available? Do you think Z should be available?
- Do you think X should be available?
- Do you think Y should be available?
- Do you think Z should be available?
I would like to know if there is a significant difference in respondents' beliefs in the availability of X and the availability of Y. Here is an example crosstab:
(N.B. most of the responses to the questions are much less unanimous than this).
When I do a chi2$\chi^2$ test in Stata (I did not use Fishers due to sample size), it says there is a significant difference with a p-value of 0.00000000000000001114 (!!!). This seems rather unlikely to me (just look at the values!). I tried Stuart-Maxwell using the symmetry command and got a much more reasonable p-value of 0.0007. I used Stuart-Maxwell instead of McNemar as it is not 2x2.
Should I use chi2$\chi^2$ (as I have been doing)? I feel like I shouldn't be. Also while the data is not unpaired it is not repeated measures, it is asking respondents about their attitudes to two different things and I would like to know whether there is a significant difference.
Which is the most appropriate test to use? One of these or some other test I haven't considered?
