I'm trying to understand standard deviation. I understand how to determine standard deviation across a group, but I am not understanding using standard deviation to compare groups. Am I attempting to use the correct tool?

For example, group 1, 2011, for 10 people, these are the results:

99 98 91 89 88 74 74 72 67 56
average = 81
Variance = 18 17 10 8 -7 -7 -9 -14 -25

324 289 100 64 49 49 81 196 625

1777/10 = 178 so stdev = 13

Group 2, 2012, for 10 people, these are the results:

99, 91, 90, 88, 87, 86, 84, 83, 74, 65
average = 85
variance = 80
stdev = 9

1) If we assume that the tests were similarly hard, but may have slightly different questions, would it be correct to assume both students with 99 have the same ability? Or should I be comparing apples to apples by adjusting both sets of scores to a set average, say, 80?

If I need to adjust, what do I do to compare fairly, and what is it called?

  • 2
    $\begingroup$ Is this a self-study question by any chance? $\endgroup$ Jan 9 '14 at 12:12
  • 1
    $\begingroup$ yeah learning myself off internet khan academy etc $\endgroup$
    – sayth
    Jan 9 '14 at 12:53

You have two options:

  • In case you assume the tests were similar:

Pool the test results, so you have twenty tests results with a mean and standard deviation. In that case, both students with score 99 have performed equally.

  • In case you assume that one of the tests was harder than the other:

If the professor set up two tests, it might still be that he under- or overestimated some questions. If at the end of the exams, he sees that the means of the tests are not equal he can correct for this variance "across" tests. He will then use only the variation "within" tests to give students a final score. In that case, both students with 99 will not score equally on the tests.

In case you would like to correct for this variance across groups, you have many options but consider these easy ones:

  • Just correct for the average difference, i.e. subtract the difference in average for each score within a group;
  • Assume a distribution in each group, and calculate the standardised scores (i.e. z scores);
  • Forget about the distribution in a group and calculate the rank of each test within a group. (the highest score in each group gets 20/20, the median score 10/20 and the lowest score 0/20).
  • $\begingroup$ how do.I adjust for variance? $\endgroup$
    – sayth
    Jan 9 '14 at 12:57
  • 2
    $\begingroup$ I edited my answer to answer on your comment $\endgroup$
    – Kasper
    Jan 9 '14 at 13:04

If you assume the tests are equally difficult, then the students in 2012 were slightly smarter than those in 2011 and you can safely compare the students across years.

That, however, is an assumption.

If, on the other hand, you assume that the students in the two years were equally smart, then the two tests were slightly different from each other and you can compare students across years by turning all scores in both years to z-scores, based solely on their year.

That, also, is an assumption.

If you are unwilling to make any assumptions, then you don't have enough data to compare students across years. This is a field of psychometrics called test-equating. It's quite involved, but was designed to solve exactly this problem, e.g. when comparing scores on multiple versions of some standardized test that is used to make important decisions (like college admission or licensure in some field).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.