Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I have a survey with 25 items that were administered pre and post an intervention. Each item had four possible responses:
The survey was given to a class of 60 students. I have entered the data into SPSS and created a case for each student, with Pre and Post responses. As the data are ordinal in nature it seems like I should use a non-parametric test. Is the Wilcoxon-Mann-Whitney test appropriate?
Thanks.
I imagine that your dependent variable will be a composite of some or all of the 25 items. If this is the case, your composite variable can take
$$kc - c + 1$$ categories, where $k$ is the number of items, and $c$ is the number of categories. Thus, if you formed a composite based on the 25 items, your scale would take on $25 \times 4 - 4 + 1 = 97$ different values. In most cases, researchers treat such scales as numeric variables. This of course is an assumption, but I generally find it a useful assumption and certainly more useful than using non-parametric tests.
Thus, a paired-samples t-test would generally be appropriate assuming at least a reasonable spread of scores on the scale.
If you are happy summing several items (or taking the mean or otherwise using the arbitrary codes to create a composite score), you already assumed more than merely ordinal measurement so you might as well go for the t-test.
On the other hand, if you want to be absolutely strict in treating your data as ordinal, the Wilcoxon signed-rank test is not particularly helpful because it is based on a ranking of the differences between pre-test and post-test scores, which means assuming than $2 - 1$ is the same as $4 - 3$ and therefore than the codes you assigned to each response have some meaning beyond their order (see Is ordinal or interval data required for the Wilcoxon signed rank test?).
In fact, the Wilcoxon signed-rank test is most useful with measures that are continuous and interval-level but have poorly behaving distributions (skew, mixtures of normal distributions, etc.) In that case, it can have much better power than the t-test. So you might want to consider using it with scales' scores (composite of several items), not as a way to avoid assuming that your data are measured on an interval scale but to increase power.
If you are looking for a simple alternative, item-by-item sign tests would avoid any such assumption. However, with a relatively small sample, many items and very few categories, ties are going to be a problem, multiple testing issues, power and interpretation too.
All of these approaches (and some others) can be defensible but the popular notion that interval means t-test and ordinal means “non-parametric” is not a good way to guide analysis.
