0
$\begingroup$

I am new to this site. A greenhorn at designing surveys, got a data collection question.

I designed a question to gather what individual thinks with regards to doing X at different locations L1, L2, L3 ... etc. My options for each location L, laid out as a matrix, are as follows e.g.:

  • I am morally obliged to do X
  • Everyone does X, I just follow.
  • I am required to do X.
  • I am expected to do X.
  • I feel I should do X.
  • I need not do X.
  • I should not do X.
  • I must not do X.
  • X should be done for me.

One of the responder commented "please learn how to use a Likert Scale properly". Since the survey is anonymous, I cannot seek him for more advice.

So I went on to read about Likert Scale, to see what defines a Likert Scale. Now, I can see clearly my matrix of radio buttons is not a Likert Scale. I then went on to find what other kinds of (named) scale are there in this subject, hoping to find one that may fit what I just did, but everywhere I go is another Likert Scale.

Question: Is Likert Scale the only way to collect data with a matrix of radio buttons? What other scales are there? If Likert is the only scale, how should I modify my question?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
3
$\begingroup$

Not exactly an answer, but probably may help moving this along. The question is treading between two different areas and I'll comment on for each of them.

First, there are other alternatives to Likert's scale to represent the respondent's most fitting choice. At its roughest dichotomous form (yes/no) to finest continuous form (visual analog scale.) Though I am not sure if that's the answer you need.

Second, a bigger problem I can see is that your scale is not really suitable for radio button type of pick-one-only format. It's because the items are not exclusive of each other. For instance, morally obliged may also mean I am expected and/or I am required.

I would suggest i) consulting relevant literature on the topic you're looking at see if someone has already developed a test. You can then consider adopting it. For example, here is a test development paper about work ethics. ii) Consult someone who is familiar with psychometric and get help on flushing out your concepts and underlying constructs. Once they are clearly expressed, the format of collecting responses should be more apparent.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for your comment. It's a great start for me. However, I just want to point out that, when developing the question, I wasn't thinking of creating a scale at all. The options are distinct responses and seek a "best fit" response. For example, if I have the multiple choices, "< 25%", "< 50%", "< 75%", "< 100%", one can see that "< 25%" is also "< 100%", but "< 25%" is more precise than "< 100%". I am seeking a "chose the most correct" response than a scale. Does it make sense? Do I still read about scales if this is the case? $\endgroup$
    – Jake
    Mar 14, 2014 at 1:24
1
$\begingroup$

There are other ways to create responses scales to questions. For example, your scale option in your follow-up comment is closer to what is called a Guttman scale. You should most certainly about a book about survey construction and measurement. In the posted question, the items do not appear mutually exclusive, so you cannot place any given person on a single continuum. For example, how do people differentiate between feeling "required" and feeling "expected?" The difference might be clear in your mind, but respondents are likely to have different interpretations and ways of responding. This introduces error in your data.

In addition to the technical issues of survey construction, you will also want to have strong knowledge of validity theory and reliability. These represent the foundation of measurement, and the quality of the research is entirely dependent on the quality of the measurement. Garbage in, garbage out.

The issue of radio buttons is really about the technical implementation of the measurement. Radio buttons can be used on an electronic survey -- in fact, it is commonly done for Likert-style response options. They may or may not be the best option depending on your overall formatting and the number of response options.

Improve this answer
$\endgroup$
5
  • $\begingroup$ +1 Guttman scale. But I don't understand why required and expected cannot be distinct. Firstly, they are different, I don't think this needs elaboration, or do you sincerely cannot see the difference? But like I said, the purpose is for the respondent to answer based on their definition, not based my definition. So given a scenario in which they determine their response, the results will show how many think it is required vs. expected. eg. Likert scale of painfulness, someone's "6" may not be the same as another person's "6", so similarly, it is based on their definition. No? $\endgroup$
    – Jake
    Jun 19, 2014 at 1:35
  • $\begingroup$ Also, the fact that expected and required are separated, is a hint to the respondent to try to distinguish between the 2 get a "best fit" answer. Again, as this is to get their opinion by their definition, it does not matter if it is "wrong" by my definition. Hope you can explain or suggest references for me to understand more, thanks in advance! $\endgroup$
    – Jake
    Jun 19, 2014 at 1:44
  • $\begingroup$ @Jake If everybody is allowed to make up their own definition of a concept or apply their own interpretation to a scale item, what are you measuring? And what does it mean if my six is added to your six, but our own sixes have fundamentally different meanings? You can't add apples to oranges. $\endgroup$
    – Brian P
    Jun 19, 2014 at 1:44
  • 1
    $\begingroup$ @Jake And, look at the differences between ratio / interval levels of measurement and compare the properties of those levels with ordinal and categorical. This, along with theory of validity, determines what can and cannot be added together. $\endgroup$
    – Brian P
    Jun 19, 2014 at 1:46
  • $\begingroup$ I am not challenging you, only just asking. That's what I am saying cannot add apples to oranges. Hence, I am just getting how many people think it is required or expected. For example, I put up a green-brownish colour, then can I not ask people to choose whether it is green or brown, just to see general tendency towards color? +1 for theory of validity $\endgroup$
    – Jake
    Jun 19, 2014 at 1:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.