From a survey I have data on three similar Likert-type questions (scales are 1 to 5). Because the questions essentially ask the same thing, I'd like to just report the average (mean) score across all three questions*. How do you calculate the margin of error for that?

  • Do you just lump the data from all three questions together? But then you effectively triple your sample size, and you get a relatively small margin of error.

  • Do you calculate the average of the three responses for each respondent first, and then calculate the margin of error? But does that reduce your standard deviation?

  • Do you take the largest margin of error of the three questions? I heard somewhere that one typically reports the largest margin of error. Does that apply in this case?

  • Or is it something else?

Thanks in advance for any advice.

*PS: I know about the controversy of treating Likert-type data as continuous variables. In this case I have no choice but to accept that.

  • $\begingroup$ How does lumping the items triple your sample size? Instead of three items each ranging from 1 to 5 you have one item ranging from 1 to 15 (if you sum them) or 1 to 5 (if you take the mean). That sum or sum will have all the usual characteristics - mean, sd, skew, kurtosis etc. $\endgroup$ – Peter Flom Nov 29 '12 at 11:29
  • 2
    $\begingroup$ Note that the variance of the mean of several variables with equal variance will indeed be smaller than the variance of the original variables. That's the whole point of multi-item scales and the basis for much of classical psychometrics. The underlying idea is that the item-specific variance that gets “lost” in the process is error variance that isn't related to the characteristic you are trying to measure and should be averaged out. Therefore, adding appropriate items gives you a more precise measurement, more powerful comparisons between groups, etc. $\endgroup$ – Gala Nov 29 '12 at 12:27
  • 1
    $\begingroup$ Thanks a lot for the comments! Peter: makes sense. @Gael: That's a really interesting point you make. Actually, while thinking through your answer, I realized that the three questions are not really measuring the same thing, but rather 3 different components/aspects of it. Would that change anything? $\endgroup$ – John Dec 1 '12 at 11:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.