2
$\begingroup$

I want to build a standardised score on ordinal data. Data comes from several scales from a personality test of summed up rating items. It seems that ordinal data in psychology is often treated like being from a higher scale of measure.

My questions:

  1. Is it usual practice to build z-scores and percentiles for psychometric tests from ordinal data, as they are based on the mean an standard deviation?

  2. Will "purists" be okay with building z-scores and percentiles like this? Is there some "trick" (e.g., McCall surface transformation) that should be used instead to make everybody happy?

  3. Is it usual practice to use a T-Test on this kind of data? The test seems to be very robust. But is it not "better" (e.g., safer) to use Mann-Whitney U-Test or related tests instead?

  4. I'd say I'm actually not a purist, as I understand that there are some advantages when treating ordinal data like being from a higher scale of measure. But where are the limitations of good and usual practice doing so.

$\endgroup$
1
  • $\begingroup$ It's worth pointing out that there's at least one school of thought that would suggest it's only appropriate to treat such summed ordinal scores as interval data if the individual responses fit an IRT model (e.g. the Rasch model) $\endgroup$
    – Ian_Fin
    Sep 12 '16 at 20:25
2
$\begingroup$

It is widely used practice to use so called Likert scales as quasi metric in a school of thought called Classical Test Theory (CTT, as opposed to Item Response Theory, IRT).

Usually you are expected to formulate the possible choices so that they feel equidistant and you are supposed to prove that what you add is somehow of the same kind and not apples and pears (think: Internal consistency, think: factor analysis).

If your scale has Internal Consistency and is composed of a number of items and otherwise "reasonable", then it is common practice to compute t-tests and do all sorts of other stuff for metric variables.

It may not be mathematically correct but is has stood the test of time, and often results are surprisingly similar compared to those gained by more complicated and less used IRT, even if the latter may be the future in a computerized time.

To read more, search for "Likert-scale" and "Classical Test Theory".

$\endgroup$
1
  • $\begingroup$ Great! Seems like most books and tutorials just proceed like this without loosing to many words about it. Thanks for clarification! $\endgroup$
    – Stats Doe
    Sep 13 '16 at 18:53
2
$\begingroup$

Will "purists" be okay with building z-scores and percentiles like this?

No, they will not. The use of higher-order measurement scales on data from lower-order measurement scales means that operations are being used that are not invariant to valid re-labeling of values in the lower measurement scale. For example, if you have ordinal data then it is labelled in such a way that the labeling can be altered by any strictly increasing transform, and you still preserve the same ordering of values. If you build z-scores from this data, this involves use of the mean and standard deviation, which are not invariant to valid re-labeling of the values. This means that your z-scores will be affected by arbitrary labeling choices.

Now, it is necessarily the case that a "purist" will be someone who expects you to apply the theory in a pure manner, without fudging it by using operations that are not allowable in the theory. Indeed, one might even go so far as to say that a measurement-theory purist is defined by the fact that they are not okay with you doing this. So no, you cannot deviate from the theory and then expect a "purist" to be okay with this.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.