# How to check if two lists of rankings lists are statistically significant different

To make my problem more clear I will give some examples of my data:

Measure A Measure B
Set 1 5 3 1 4 2 5 4 2 3 1
Set 2 5 1 3 4 2 5 4 1 3 2
Set 3 2 4 3 5 1 5 2 3 4 1
Set 4 3 4 1 5 2 5 3 2 4 1
Set 5 5 3 2 4 1 5 4 2 3 1

I have multiple sets of items to be ranked and multiple measures by which the items in a set can be ranked. Each cell in the table presents a ranking of the items in one set by one of the measures (e.g. rankings of the 5 items in set 1 as ranked by measure A are in the cell in at line 2 column 2 of the 3x3 table).

My goal is to check if Measure A ranks items differently than Measure B. I want to check if there is a statistically significant difference between the way that the two measures ranked the items in the different sets.

While searching I only found ways to compare 2 rankings between them (e.g. comparing the ranking of measure A on set 1 and the ranking of measure B on Set 1), but not ways to compare multiple rankings lists.

• I am not clear what you would count as an insignificant difference. You can see they do produce different rankings, as all of you sets show differences and you only needed one such set to say this Commented Dec 5, 2021 at 16:41
• What it sounds like to me is that you'd like to construct a distance metric on $S_n$. This means a quantitative way of computing "how dfiferent" two permutations of $n$ items are. I recommend looking into a few different distance metrics on $S_n$ to find which one best fits your application. EDIT: Here's a helpful survey of some metrics: cicirello.org/publications/cicirello-bict2019.pdf Commented Dec 5, 2021 at 16:57
• I am sorry, english is not my first language and maybe I did not explain correctly. I want to try and prove that Measure A ranks things differently than Measure B. I can't just say that they are kind of different sometimes. I have 300 sets of items that can be ranked, and later there will be even more. I took a sample of 40 sets to analyuze and I want to use that sample to show that, in general Measure A is different form Measure B, not coincidentally sometimes different. Commented Dec 6, 2021 at 10:31
• Like when you measure the blood pressure of patients before and after treatment you want to prove that the mean indeed changed significantly, and you need the paired data t-test to prove that, you can't just say that they seem different. I was asking what kind of test or design I could use to prove that statistically, with a p-value and so on. Commented Dec 6, 2021 at 10:31