2
$\begingroup$

Suppose a survey has five items with a six-point Likert scale. It has been rolled out with ill-formed labels:

*the (2) was labelled by "somewhat disagree" instead of "disagree" - and vice versa the (3) was labelled by "disagree"

*analogous ill-forming for (4) and (5): "somewhat agree" and "somewhat disagree" were mixed up

Taken literally, the respondents face a 132546-situation instead of a 123456-situation for some of the items. The begin of the survey was labelled right.

Can this be repaired by a clever and statistically sound repair-procedure as part of the data analysis, if the answered questionnaires should form a basis for structural equation modelling?

From my point of view:

clearly no, the questionnaire must be sent out once again in a sound shape. Even if there is a general conditioning of answering this questionnaire as of the labels were in the right order, "conditioning" is not a 100% concept. So, there are two things we would not know:

how many people took the questionnaire literally and who took it literally.

This introduces a "second noise" to the structural equation model, which uses p-values etc. to quantify "noise". Now, "noise" and "second" noise cannot be separated anymore. Even if we knew the relative amount of conditioned answering, we still did not know, who answered in an unconditioned fashion - ending up with the non-separable "second noise" again.

Normally I am really not that harsh. Thatswhy: any objections to my point of view - is my position too harsh?

Improve this question
$\endgroup$
1
  • $\begingroup$ maybe to add a small provocation: one could "post-test" the surveys, and if there is a 100%-conditioned behaviour among the participants in this post-test, confidence intervals would be with an error = 0, as p^=100% . This works through the formulas I found, but I do not like it. $\endgroup$
    – Statos
    Commented Jul 10, 2014 at 11:02

1 Answer 1

Reset to default
1
$\begingroup$

I would say: no, this damage is not repairable. SOME people will have taken the numbers literally, some will have used their training and interpreted it as 123456. But in most cases, the order will have caused confusion which might influence the persons in rather unpredictable ways. This is a classic example for GIGO - garbage in, garbage out. There is no way to proudce reliable results from this.

Improve this answer
$\endgroup$
1
  • $\begingroup$ However, a final provocation: let's make a 4-Likert scale from the one above. As the extremes of the scale were chosen right. Counterargument similar to Christian's: e.g. maybe (and we'll never know, if that happened or not) a higher spread of the scores in comparison to other answers happened due to the Irritation introduced. $\endgroup$
    – Statos
    Commented Jul 10, 2014 at 14:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.