# Test whether two rank orders differ

I have 20 human subjects doing a task under 2 different difficulty conditions. (So it's a repeated measures design)

Each person has a different performance in each of the task difficulty conditions, so I can rank-order them. I want to test whether there are differences between the ranking of performaces between conditions.

I'm essentially looking for a test that would give me a significant result when there is no correlation between measures.

EDIT: To clarify, I want to see if the rank order 1 2 3 4 5 6 7 8 9 10 is different from the rank order 1 3 4 5 2 8 9 6 7 10

The Wilcoxon signed-rank test -and other rank-based statistics- won't do, because their null hypothesis (unless I'm mistaken) is that the median of the two ranks is equal. This allows to test overall differences in rank order (when the two conditions are ranked together). What I need is a test that will compare whether the order itself changes.

Can anybody point me to a test that has the null hipothesis that the rank orders are equal?

Thanks

• @ Zach: Thanks. The null of the Wilcoxon signed-rank test (unless I'm mistaken) is that the median of the two ranks is equal. This allows to test overall differences in rank order (when the two conditions are ranked together). What I need is a test that will compare whether the order itself changes. I.e., I want to see if the rank order 1 2 3 4 5 6 7 8 9 10 is different from the rank order 1 3 4 5 2 8 9 6 7 10 Dec 13, 2013 at 18:33

So if I understand you correctly, is the question you are trying to answer if there is a difference in the way individuals change across the two tests relative to one another? If so your null hypothesis would be the response of individuals is the same. You could test this using the actual values instead of their ranks and running a linear mixed model with random slopes and compare that with the model without the random slope:

lm1<-lme(performance ~ test, random = ~ test | individual, data = data)
lm2<- lme(performance ~ test, random = ~ 1 | individual, data = data)
anova(lm2,lm1)


If model lm1 gives a significant better fit it shows individuals differed in the way their performance changed across the two tests.

If I understood correctly, you want to test if a condition affects the score order of a population. I guess what you are looking for is the Friedman test or its generalization. On wikipedia : http://en.wikipedia.org/wiki/Friedman_test. I think the example with the judges and the wines looks very similar to your problem.

I think the wilcoxon signed-rank test will work in this situation. The null hypothesis is that the 2 sets of ranks are equal.

• The null here (unless I'm mistaken) is that the median of the two ranks is equal. I'm not interested in the mean of the rank for 2 conditions, but in the rank order itself Dec 13, 2013 at 18:30
• @elisa You might want to edit your question then. What you're asking for is currently not clear to me.
– Zach
Dec 13, 2013 at 22:08