2
$\begingroup$

Background: I'd like to use a 7-point Likert scale for an upcoming survey (focused primarily on opinions). For the most part, the questions will be centered around the "To what extent...?" stem. That is, the answers will range from, e.g., "To a very small extent" to "To a very great extent". Using the 7-point Likert scale, it is most often recommended to include the "neutral" as the center/middle score. Based on my research thus far, however, I'd like to discourage people from not taking a stance on the subject. That is still debatable though. Now, to ensure I'm in no violation of "knowledge liability" (i.e., respondents must know the answer), it was recommended to include the "N/A" in the question/scale. In my view, "N/A" does not equal the neutral. So, I don't think the center position would be the right location on the spectrum.

My questions:

  • If I were to include the neutral in the survey questions, what's its location in the scale?
  • How should N/A answers be numerically coded?

If it would be the most-outer left point (code value = 1), it would take the place of the "To a very small extent"... which then has been shifted by 1 score to the right (code value = 2) and thereby isn't the total opposite of the "To a very great extent" (code value = 7).

$\endgroup$
5
$\begingroup$

If you give it a place in the ranking of your likert scale, this is just as arbitrary as making it equivalent to "neutral". Clearly, this is not the way to go.

The cleanest way to handle this type of situation is probably by way of missing data algorithms. Might I recommend that you read "Statistical Analysis with Missing Data" (by Little & Rubin)?

Bottom line, in your case (from what I read above - maybe I'm missing valuable information), I suggest using a multiple imputation method. Imputation amounts to replacing all missing values (N/A) by random (valid) values. This results in a completed dataset that you can use as normal, so you can perform any analysis you would. In multiple imputation, you repeat this a number of times, and then you average the results over your repetitions. In many circumstances, this correctly accounts for the missingness and can be quite efficient. In addition, a formula exists (see the book I mentioned) to correct the variances, so you continue to have valid inference?

$\endgroup$
4
  • $\begingroup$ Nick: Thanks for the prompt response... and the recommendation to look into the Little/Rubin's publication. $\endgroup$ – user22000 Mar 14 '13 at 16:25
  • 4
    $\begingroup$ I'm not sure than the N/A here is actually missing - it seems like the respondent is saying 'these answers don't apply', not 'My answer is one of these but I can't discern which one.' $\endgroup$ – Glen_b Mar 14 '13 at 22:41
  • 2
    $\begingroup$ Perhaps you are misled by the term missing: it does not imply that you had the answer once and it somehow got lost. For most situations, missing data just means: for some (unknown) reason, the scientist does not know what goes there. A two-step model, like you propose in your answer, does nothing but "visualise" the missing data mechanism. Of course: if this is of interest to you, that may be a good idea, but it will likely require you to use a parametric model for the missingness. $\endgroup$ – Nick Sabbe Mar 15 '13 at 7:42
  • 1
    $\begingroup$ I'm saying that "doesn't know what goes there" isn't a correct characterization of the NA's in the OP's discussion. $\endgroup$ – Glen_b Mar 15 '13 at 12:23
4
$\begingroup$

This is not really a likert scale, not that I care about the terminology as much as many people do.

What is important though is to udnerstand why it is different. A genuine likert scale is symmetrical and has a natural central point - for example, half way from strongly disagree to strongly agree. Think of that as the zero.

In your scale, the options are strictly ascending from a small extent to a large extent. Your "zero" is at the far left of the scale, not in the middle. So the whole concept of "neutral" is flawed when it comes to a scale such as this.

I'm not saying this is the wrong scale to use, just that the question "where does neutral go" does not make sense.

$\endgroup$
3
  • $\begingroup$ Thanks... my providing the most out left/right values may have been misleading. So, yes, you're absolutely right, a Likert scale is well-balanced. And the Extent scale is exactly that. It ranges from: To a very small extent; To a small extent; To a moderate extent; [Neutral]; To a fairly great extent; To a great extent; To a very great extent. $\endgroup$ – user22000 Mar 15 '13 at 12:09
  • $\begingroup$ Still need to know more about the "N/A" option. $\endgroup$ – user22000 Mar 15 '13 at 12:11
  • 2
    $\begingroup$ I disagree that that this is "well balanced". "moderate" is not to the left of a central point in the way that "fairly great" is to the right of it. This is fundamentally different from a likert scale. $\endgroup$ – Peter Ellis Mar 16 '13 at 4:41
3
$\begingroup$

Since the N/A doesn't fit into the ordering, it might be best to conceive of it as a two stage process - one where people decide if they can give an answer on the scale (or give N/A) and then one where people who do give an answer on the scale then decide which one to give.

This would result in a two-part analysis - characterizing the ones that give N/A, and then for those that don't, looking at how they answer.

If these are DVs, then you could have a logistic or probit model for the decision to answer with NA, and then whatever you'd normally have used with your ordered categories for those that answer.

If these are IVs, you might have a dummy for the NAs and then whatever you'd normally do with your ordered categories for the others.

$\endgroup$
3
  • $\begingroup$ Glen: Thanks -- that would be ok if there's one general question that might focus on, e.g., "If you've been pregnant, please answer the following questions... XYZ". Thus, all male would select N/A. However, this is not my approach, I intent to ask roughly 30 questions covering different subjects. In some instances, N/A may be the correct answer... for others, they will have experience or an opion. Cont'd below. $\endgroup$ – user22000 Mar 15 '13 at 12:16
  • $\begingroup$ cont'd: I can't see to double the answers (30 to 60) by prefacing every single question with N/A. There must be a way to handle the N/A in a Likert scale (while maintaining the balance between left/right extremes). Final thoughts? $\endgroup$ – user22000 Mar 15 '13 at 12:19
  • 1
    $\begingroup$ Don't know how I missed this comment before. I didn't propose 'doubling the answers'. $\endgroup$ – Glen_b Feb 19 '14 at 5:35
0
$\begingroup$

I was involved in a survey team that included a Likert scale about quality of service delivery. We had a long discussion on this issue.

One aspect of Likert surveys is that we want responses for each question. However based on survey ethics that is an unfortunate expectation. Any respondant has the right not to answer a particular question. Due to this all our Likert Scale questions had a N/A option (defined as being 'not applicable' or 'no answer'). Such responses were reported in the evaluation yet were not included in the analysis of the valid responses. We simply treated the N/A responses as being outside survey. This of course has implications in terms of confidence intervals and effect sizes etc so its potential impact should not be underestimated but it is a more open and honest way to treat the responses with ethical consideration.

In particular, there was the risk that an N/A option could have been over used by respondents (similar to the over use of the neutral option). However we did not find this to be the case. In fact, one thing we noted was that this approach totally eliminated missing data which often occurs because respondents don't want to answer a particular question. Hence providing a N/A option probably improved the robustness of our survey as it gave more freedom to the respondents.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.