# Which statistical tests can be used to compare two paired/related rankings?

To understand my question, can anyone to evaluate if the statistical analysis in the below research (from the 2011 year) was performed correctly? I must to analyze a similar research and I am interested in the correct method. Initially I supposed that in this kind of research must be used the Wilcoxon test, but now I am inclined to the Spearman correlation test. Is this correct? Thank you!

The most common tests for comparing means or medians of paired observations are (in no particular order):

i) paired t-test

ii) Wilcoxon signed rank test

iii) sign test

For multiple related observations, you could treat the factor that relates the observations as a block and hence use ANOVA for a randomized complete block design in place of (i) and a Friedman test in place of (ii), though such things might be treated as repeated measures, depending on circumstances.

Regarding the paper, the situation is somewhat different from that title question in a couple of respects.

Not least, there are many tests!

Further, there are a number of items than are given numerical ratings and an importance ranking.

In the analysis in the paper, they seem mostly to have used paired t-tests to compare the importance ranks of individual items. However those rankings can't independent across items (if I rate one item higher, other items must rank lower), meaning that the tests aren't independent. Indeed, the ratings wouldn't be independent either. (This of itself doesn't invalidate the individual tests, but they should probably be analyzed in a way that considers the multivariate structure of the problem.)

The main question would appear to be 'is a paired t-test suitable for this situation'?

Multiple testing aside, and ignoring the multivariate structure (with likely loss of power):

The underlying distributional requirement would be that the pair-differences are approximately normally distributed (with constant variance) under the null. We'd also require independence across subjects.

--

One final comment - they appear on first glance to be asserting 'no difference' on the basis of failure to achieve significance from a hypothesis test. The specific words are 'did not change'. Unless I missed something from my first quick skim, this is not a reasonable conclusion on the basis of the analysis.

• Thanks for your answer! I agree with your above suggestion and soon I will formulate in a new question what I want to know.
– Yuri
Commented Mar 26, 2013 at 4:44
• Also see McNemar's test. Commented Sep 28, 2023 at 23:39