# Using Standard Deviation to Understand Consensus

I was recently asked a question, and while I think I know the answer intuitively, I can't explain it.

Scenario: A questionnaire of 20 questions was given to 6 people. Each person was asked to answer each question by ranking it on a scale of four numbers: 1, 2, 3, 4, 5.

The questionnaire was then analyzed, and, for each question, the standard deviation was calculated using the 6 people's rankings of 1-4. This standard deviation was then called the "consensus" of a question. All the consensus numbers were then ranked against each other and those with high standard deviations were considered to be "high consensus."

My Issue: This doesn't make any sense to me.... at all. The standard deviation is a measure of "how much the data varies." If you look at a single point and compare it to the standard deviation, it should tell you how close to "normal" it is (for the data set you are analyzing).

A better measure of consensus would be to just add up how many people voted similarly for a single question (i.e. if 4 of the 6 people voted the same rank for one question, then there is high consensus).

Am I crazy, or does the standard deviation method actually work and I'm just not understand it?????

• I think you want to compare variances across questions. For example, say 2 people filled out the survey and q1 is (1,5) and q2 is (3,3). The variance of the first question is 8 and the second is 0, so we might say there is less consensus on the first. Jun 8, 2015 at 23:24
• @DimitriyV.Masterov But consider for five people: q1 is (1,1,2,3,4) with sd of 1.3; q2 is (1,1,1,1,2) with sd of 0.4; q3 is (1,1,4,4,4) with sd of 1.6; q4 is (3,3,3,4,4) with sd of 0.5; This implies the consensus is q2 < q4 < q1 < q3, but that's not true. It should be q1 < q4 <= q3 < q2. The standard deviation is combining the number of votes with the scale of the vote, but that doesn't show consensus. Jun 8, 2015 at 23:50
• Please add the [self-study] tag & read its wiki. Jun 9, 2015 at 0:01
• @gung This is not a question from a book or for a course. This is a real life scenario I'm currently faced with. A coworker created a mess of a document, and I'm trying to sort through it. I feel like my logic is correct and that the standard deviation is being incorrectly used. Given that, I'm happy to add [self-study] if you feel it is appropriate, but I'm really trying to just work through this logic. Jun 9, 2015 at 0:08
• @VeryConfused I would not defend this method very hard, but that seems to be what you colleague is doing (other than the part about high variance being high consensus and why only 1-4). In any case, your ranking is not entirely obvious to me either. Jun 9, 2015 at 0:52