I have seen some analysis of a survey where responses across a dozen similar questions which identified perceived problem issues have been weighted to create a composite score. Possible responses to all 12 questions were "not an issue", "minor problem", "medium" and "major problem" with a known % of respondents against each of the 4 responses. The composite was created by multiplying the first category by (-1), "minor" by (+1), "medium" by (+2) and "major" by (+3) - net result a typical score of between 0.5 and 1.5. The analyst has then ranked the questions by the highest score to assert these were the most important of the dozen "issues"

Is this a known and well used device? Does the ranking not potentially change if you say "not an issue" = zero, instead of (-1)? Why (-1, +1,+2, +3) - again would the ranking not change if one used say (-2, +2, +4, +6)? As I have never seen or heard of this before I would welcome views as to whether it is real and valid (and/or well used anyway) or spurious ???

sorry if I wasnt clear

there were 12 questions and respondents were asked to identify which mattered to them, and if so, then to what extent - you could tick "not important to me", "matters little", "matters medium" or "is very important" and tick one box only but you had to tick one of those 4 responses.

Each percentage of the total responses per question was then multiplied by -1, +1, +2 +3 respectively, creating four sub-totals which are then aggregated to create an overall composite "score" for each of the 12 questions which then permitted ranking. Max possible score could be 3 where 100% said it was mucho important (ie 100% x 3), min possible score is -1 where no-one thinks it important (ie 100% x -1)

I have seen +1, +2, +3, +4, +5 as subs for Likert "disagree strongly", "disagree", "neutral", "agree", "agree strongly" to create 'totals' but never seen a negative weight..........

The argument given was the negatives of "dont care" pretty much directly offset the lowly "matters little" but the "matters medium" and "matters mucho" are given increasingly important emphasis by scaling to double, then triple.

My gut says arbitrary and hard to justify - has anyone seen it or similar before? They are not dependent v independent in any meaningful quantitative study - it was to identify which of the 12 itemised characteristics mattered most to the respondents - such as "the operating environment is cold" or "your boss is an ignoramus"

  • $\begingroup$ You mean "ranked the questions by the average score across respondents"? $\endgroup$ – Scortchi Jun 1 '15 at 11:49
  • $\begingroup$ State whether such questions are used as dependent variables vs. independent variables. There are wonderful solutions if they are dependent variables (ordinal regression) without assuming codes and only assuming ordering. $\endgroup$ – Frank Harrell Jun 1 '15 at 12:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.