I was recently asked a question, and while I think I know the answer intuitively, I can't explain it.
Scenario: A questionnaire of 20 questions was given to 6 people. Each person was asked to answer each question by ranking it on a scale of four numbers: 1, 2, 3, 4, 5.
The questionnaire was then analyzed, and, for each question, the standard deviation was calculated using the 6 people's rankings of 1-4. This standard deviation was then called the "consensus" of a question. All the consensus numbers were then ranked against each other and those with high standard deviations were considered to be "high consensus."
My Issue: This doesn't make any sense to me.... at all. The standard deviation is a measure of "how much the data varies." If you look at a single point and compare it to the standard deviation, it should tell you how close to "normal" it is (for the data set you are analyzing).
A better measure of consensus would be to just add up how many people voted similarly for a single question (i.e. if 4 of the 6 people voted the same rank for one question, then there is high consensus).
Am I crazy, or does the standard deviation method actually work and I'm just not understand it?????