2
$\begingroup$

I'm conducting a user study where I ask respondents to rank different versions of same sentences by their preference. What would be an adequate way to analyze this type of ranked data? Ultimately, I'm interested in seeing which version of sentences are most/least preferred but I'm not sure what kind of tests I need to be using.

Also, if there are other types of information that I can collect from such ranked responses, could you please let me know?

Improve this question
$\endgroup$
4
  • $\begingroup$ Maybe you can use order statistics to test the distribution of your data? Rank the sample data from smallest to largest or viceversa and the kth value is a random variable, say Y_k, so it has a distribution, e.g., you will have the Max, Min, 3rd order statistic, etc. . This distribution depends on the distribution of the underlying data/process, e.g., the kth order statistic from a N(0,1) ( Normal, mean 0, Variance 1) will differ from the kth order statistic from, say, an exponential. If you want to test whether different users have the same preferences you can use the Wilcoxon rank test, $\endgroup$
    – MSIS
    Commented Nov 4, 2020 at 2:40
  • $\begingroup$ which is non-parametric i.e., you don't need any special assumptions on the underlying distributions. But it depends on what your end goal is. Hope I understood your point well, let me know otherwise. en.wikipedia.org/wiki/Order_statistic $\endgroup$
    – MSIS
    Commented Nov 4, 2020 at 2:41
  • $\begingroup$ which is non-parametric i.e., you don't need any special assumptions on the underlying distributions. But it depends on what your end goal is. Hope I understood your point well, let me know otherwise. en.wikipedia.org/wiki/Order_statistic or the Mann-Whitney Sum test duckduckgo.com/… $\endgroup$
    – MSIS
    Commented Nov 4, 2020 at 2:47
  • $\begingroup$ Related thread: Multinomial choice with binary observations. There are some extensions to the case of k-item ranking. $\endgroup$
    – chl
    Commented Nov 4, 2020 at 8:57

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.