1
$\begingroup$

I'm looking at the appropriateness of using the mean to summarize some Likert-type items, as opposed to taking some factor analysis approach. Ideally I'd like to use the mean, since it would be the easiest solution.

There are several (1,2,3) threads on related topics, and the general message I got from them was that researchers generally use the mean, although the appropriateness of doing so is disputed.

My question is different, and relates to how I can tell whether it's appropriate to use the mean given particular aspects of the survey under consideration.

What are the most appropriate considerations that could drive such a decision? I've made a list of candidates below, but I am not sure whether they are sensible ones, or whether my suggested ways of assessing the considerations are appropriate.

  1. Look at whether the items are correlated with each other. I could perhaps throw out items with near-zero correlations, and investigate whether items with negative correlations with the rest have been miscoded.

  2. Look at the response format of the items. As it happens all the items in my survey have the 4-point format "Rarely, Not often, Often, Mostly". Presumably I would be happier using the mean as a summary measure if I had a 5+ point format, and if it used a more traditional Likert format with a neutral middle point, e.g. "Strongly disagree, Disagree, Neither agree nor disagree, Agree, Strongly agree".

  3. Compare the mean score on the items with some other measure of the outcome, if I have one. I'm not sure what formal tests I'd do, but perhaps I could construct a Bland-Altman plot of the agreement between the measures.

  4. Subjectively assess the items to check there aren't wild differences between them in terms of how related to the outcome they seem/how much they should be weighted. @Scortchi mentions a nice example here:

If your items are "I sometimes enjoy a salad for lunch", & "Meat is murder", what does the average score measure? Propensity to vegetarianism? Probably not: a score of 4 for the former & 2 for the latter will be a much weaker predictor than vice versa.

Improve this question
$\endgroup$
2
  • $\begingroup$ Wouldn't this highly depend on what you want to know? Positive, negative and no-correlation between questions can all be relevant. One of the reasons to not use the mean when you only have 4 options is that there is less data (only 4 options), so there is less reason to summarize. I assume you at least want to add the standard deviation. That would reduce the data from 4 numbers to 2, is that reduction essential? $\endgroup$
    – dimpol
    Oct 24, 2016 at 11:54
  • $\begingroup$ The questions are all meant to be measuring the same thing, although I haven't undertaken any independent verification of that. Often in contexts of this sort it's considered desirable to have a single summary measure for each subject. If I was to forgo that, would you suggest that each subject has four scores - i.e. "Subject's number of Rarely responses", "Subject's number of Not often responses", "Subject's number of Often responses", "Subject's number of Mostly responses"? $\endgroup$
    – user1205901 - Слава Україні
    Oct 24, 2016 at 12:21

1 Answer 1

Reset to default
1
$\begingroup$

As a preliminary step, given that your answer format is:

Rarely, Not often, Often, Mostly

You could transform it to a proportion in some reasonable way. E.g:

Rarely = 0.1 Not often = 0.3 Often = 0.5 Mostly = 0.7

You could play with different schemes to see if it makes a difference, I doubt that it would.

One thing you could do is do the factor analysis to see if it yields one big factor or more than one. If it's more than one, then the mean probably isn't good. If there is one big factor then if the loadings (weights) are similar for all items, the FA score will not gain you much over the mean.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.