1
$\begingroup$

I have 91 items ranked by different demographic groups of questionnaire participants. The rank lists for each group are all the same length and contain identical items.

I understand that I can perform correlation measures (e.g. Kendall's Tau) or intersection measures (e.g. Rank Biased Overlap) to determine how similar two sets of rankings from two different groups are. But are there any statistical tests available to tell me which items in two lists are ranked significantly differently?

Thank you!

Improve this question
$\endgroup$
2
  • $\begingroup$ do you have lists for each participant or just the average? $\endgroup$
    – rep_ho
    Nov 8, 2021 at 16:03
  • $\begingroup$ No, a group of participants perform many pairwise comparisons (item A vs B etc). The collective output of the comparisons produces a rating and ranking of items for that group (using the Elo algorithm). $\endgroup$
    – Routh_Gilson
    Nov 8, 2021 at 17:47

1 Answer 1

Reset to default
1
$\begingroup$

For ordinal (rank based) data, kendall concordance test holds good, as it calculates a normalized score for the number of matching or concordant rankings between the two samples.

Still you can also try other rank correlation methods like:

  • Spearman’s Rank Correlation.
  • Goodman and Kruskal’s Rank Correlation.
  • Somers’ Rank Correlation.

This statistical methods quantifies the degree to which ranked variables are associated by a monotonic function, meaning an increasing or decreasing relationship.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for your help! I was more trying to determine which specific items in the lists are ranked significantly different from each other? I.e. if item A is ranked 1st by one group, but 10th by the other group, is this significantly different? $\endgroup$
    – Routh_Gilson
    Nov 8, 2021 at 14:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.