0
$\begingroup$

A co-worker has asked me to help setup a spreadsheet that is used in the analysis of customer care emails. In particular, we are looking at three categories - the assignment of meta-data, content and communication skills.

A typical example question may be:

"How well is the email tailored towards the customers questions?" "How well did the employee answer the customers questions?"

I'm not a statistician but have conducted a few surveys in the past. After seeing the initial survey, which was basically an ad-hoc collection of questions combined with varying scaling mechanisms for each question. I figured the first step would be to iterate the goals and work back from there; otherwise I'd be basically devising my own methods from scratch, which feels a little flakey. All this led me to re-design the survey using the Likert scale due to it's simplicity in implementation and analysis.

However, without a deep statistical background I'm finding it hard to validate why it is better to use a proven methodology than a collection of different scales and measurements aggregated into a single number.

How can I convince them of the benefits if using a 'methodology'/scale such as Likert rather than various choice of measurements?

$\endgroup$
  • $\begingroup$ Do you mean using a standard questionnaire? Or an ad hoc Likert scale (i.e. several Likert-type items) to measure a given outcome as opposed to single items? Or using the same rating format throughout? $\endgroup$ – Gala Jul 18 '13 at 21:11
  • $\begingroup$ I would be using three categories, with multiple questions contained within each. In each category I placed questions that relate to each other and then used a lickert type items throughout with the intention of aggregating the sum. The original survey used different scales for different questions, for example integer rankings, 0 - 1 and 1- 10. $\endgroup$ – james_dean Jul 18 '13 at 21:25
  • $\begingroup$ Strictly speaking, a Likert (no c) “scale” is a set of items (questions or statements). The distinction matters. Are you planning to analyze the results item-by-item or somehow average several ratings? $\endgroup$ – Gala Jul 18 '13 at 21:35
  • $\begingroup$ Both. I'd like to drill down to each question, aggregate questions by category and provide an overall average. $\endgroup$ – james_dean Jul 18 '13 at 21:39
2
$\begingroup$

I am not sure there is much of a statistical argument against using different ratings formats, especially if you plan to analyze the data item-by-item. Statistically speaking, detailed ratings (more fine-grained scores/higher resolution) are nicer to work with but they also need to make sense to the respondents. In fact, the rating behavior induced by the response format would seem to be the main concern.

There is a large literature on response formats and their effect on people. This is more a psychological than a statistical problem but the main risk would be inducing confusion and undesirable response patterns. Having 5 and 7-points items looks a bit messy but is perhaps not a big deal, mixing odd and even numbers of responses, bipolar (positive-negative, agree-disagree) and unipolar (not at all-a lot, presence-absence) response formats would seem especially questionable, including binary yes/no questions where it makes sense not so much.

If you want to aggregate several ratings to form a scale or composite score, several additional issues come up. If you use a simple arithmetic mean, items with more variance will have more influence on the scale score and different rating formats can in turn have an influence on that, along with the wording/content of the items.

Incidentally (and this is not necessarily an argument for doing things one way or the other), Rensis Likert did in fact create scales mixing several response formats (an almost binary “Yes - ? - No” response format and two different 5-point formats).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.