0
$\begingroup$

I'm designing a questionnaire to measure the preferences of Chinese learners, in terms of how they learn Chinese characters. The original questionnaire gave users eleven items, and each item was a pair of converse statements about learning preferences, which users were asked to pick between. For example:

  • Practising character writing is more effective than reading a character text, for my character learning.

  • Reading a character text is more effective than practising character writing, for my character learning.

This was problematic for a number of reasons, so I elected to use a Likert scale instead. My problem is, if I pick either one of the statements, and give users a five-point Likert scale asking them to agree/disagree, strongly agree/disagree or neutral, then users may be biased towards the first option.

My alternatives are, to present both statements, with a Likert scale for each, as in:

  • Practising character writing is more effective than reading a character text, for my character learning.

    • Strongly Agree
    • Agree
    • Neutral
    • Disagree
    • Strongly Disagree
  • Reading a character text is more effective than practising character writing, for my character learning.

    • Strongly Agree
    • Agree
    • Neutral
    • Disagree
    • Strongly Disagree

Or, to present only one statement, but refine the answers to reflect the gradations between the two choices:

  • Practising character writing is more effective than reading a character text, for my character learning.
    • Writing is always more effective
    • Writing is sometimes more effective
    • Both approaches are equally effective
    • Reading is sometimes more effective
    • Reading is always more effective

I'm not sure what the normal practice is though, in designing this type of questionnaire. Is there a common practice for using Likert scales to pick between two preferential options?

$\endgroup$
  • $\begingroup$ Also, if there is a more appropriate SE for this, please let me know, rather than leaving a close-vote without comment. This seemed like the best one at the time. $\endgroup$ – Lou Apr 11 '18 at 12:30
1
$\begingroup$

Tried to post comment, but it was too long, so I will post it here.

Is it important that they directly compare these two? If not, another option would be two items like this:

(1) To what extent is character writing effective for my character learning?

1=Very ineffective to 5=Very Effective

(2) To what extent is reading a character text effective for my character learning?

1=Very ineffective to 5=Very Effective

With this approach you have independent assessments of each which could then be compared directly if needed via t-test or equivalent if you prefer not to use t-tests for Likert-type items.

As another comparison, I would state it simply like this:

(3) Which is more effective for my character learning, character writing or character reading?

___ character reading

___ character writing

___ both equally effective (another possible response option)

They select one or the other; this approach allows you to find the proportion or percentage preference between the two.

Including all three items would enable a validity check by comparing responses to item 3 against the comparison of items 1 and 2 - hopefully the two analyses will lead to similar conclusions.

$\endgroup$
  • $\begingroup$ I like both ideas, they seem neater than my own solutions. I'll wait to see if other users have opinions to weigh in as well. $\endgroup$ – Lou Apr 11 '18 at 14:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.