I have a survey question that had yes/no answers only. The order of yes-no, no-yes was randomized. The results showed some order effects. Respondents tended to choose the first option. For example, they choose "no" in the "no-yes" order when the real answer could be "yes". The distribution of the answers are shown below.
Is here anyway I can estimate the true distribution of yes/no answers?
Blockquote
My suspicions with yes/no sample questionnaire is that a fatigue factor quickly sets in!
As such, I suspect that the first few answers are probably more likely accurate and consistent in kind, and should be separately tabulated.
Later answered questions are probably more likely randomly responded to and should be disregarded or given a lower weight. This could be more quantitatively assessed with subsampling by phone call interviews, for example.
Also, one could insert test questions to ascertain as to whether the person is actually reading the question at various points, especially in longer yes/no surveys.
Perhaps inserting interesting graphics/pictures/..., as the questionnaire progresses, may assist in maintaining the reader's interest, and may even result in a higher quality survey product. This could also be verified by subsampling with people who took two (plain and graphic) versions of the same yes/no questionnaire.
