1
$\begingroup$

I'm new to this and I'm trying to analyse the results of a questionnaire right now. It's a questionnaire where users get to decide how a certain feature in a program will work.

There are descriptions of three ways the feature could work and they are supposed to rank them starting by their favorite. That's the result:

+---------------------+--------+--------+--------+--------+
|       Ranking       | Rang_1 | Rang_2 | Rang_3 | Rang_4 |
+---------------------+--------+--------+--------+--------+
| FeatureDescription1 |     13 |     21 |     12 |      4 |
| FeatureDescription2 |     18 |     17 |     14 |      1 |
| FeatureDescription3 |     14 |     11 |     20 |      5 |
| FeatureDescription4 |      5 |     14 |      5 |     40 |
+---------------------+--------+--------+--------+--------+

One way I found was to assign each rank a value (1,2,3,4), and then just do it like:

(13 * value1 + 21 * value2 + 12 * value3 + 4 * value4) / sum(13+21+12+4)

Any idea what's the best way for that? Right now I just used the Rank 1 values and choose the winner based on that. Is there something wrong with that?

Thanks!

$\endgroup$
2
  • $\begingroup$ This stats.stackexchange.com/q/139015/3277 question is similar. There is an answer which reflects on just some of many possible ways to analyze ranking question frequency data. Just for your info. $\endgroup$ – ttnphns Sep 4 '15 at 10:58
  • $\begingroup$ Thanks, that's useful and I haven't found that in my previous searches. Naively I'd say I could just use the Rank 1 to choose the "winner" and only fall back on the others if there's a draw between two Feature Descriptions. Is there anything wrong with that approach? $\endgroup$ – mediocre Sep 4 '15 at 11:14
1
$\begingroup$

Is it more important to win friends (by picking their favourite) or avoid making enemies (by not picking their worst-ranked choice)? In either case, feature description #2 would be the right one.

Actually you have a stronger result for your data. If (big "if") the business benefit of a choice whose rank-counts are $(r_1,r_2,r_3,r_4)$ -- e.g. 13,21,12,4 for option 1 -- can be described as a weighted sum of rank-counts with decreasing weights ... $$ \textrm{benefit} \equiv B(r_1,r_2,r_3,r_4) = \sum_i \alpha_i r_i \\ \alpha_1 > \alpha_2 > \alpha_3 >\alpha_4 $$ ... then option 2 is necessarily the best choice.

To see this, set $\alpha_1=1$ and $\alpha_4=0$ without loss of generality, and write the benefit in terms of $(\alpha_2,\alpha_3)$: $$ B_1 \equiv B(13,21,12,4) = 13 + 21\alpha_2 + 12\alpha_3 \\ B_2 \equiv B(18,17,14,1) = 18 + 17\alpha_2 + 14\alpha_3 \\ B_3 \equiv B(14,11,20,5) = 14 + 11\alpha_2 + 20\alpha_3 \\ B_4 \equiv B(5,14,5,40) = 5 + 14\alpha_2 + 5\alpha_3 $$ Now #1 beats #2 iff $4\alpha_2>5+2\alpha_3$ : impossible because $\alpha_2<1$.

And #3 beats #2 iff $6\alpha_3>4+6\alpha_2$ : impossible because $\alpha_3<\alpha_2$.

And #4 beats #2 iff $13+3\alpha_2+9\alpha_3 < 0$ : obviously impossible!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.