3
$\begingroup$

I'm working on a meta analysis of the mean of a single variable (using arithmetic mean as the effect size, rather than the relationship between two variables). The attitude I'm assessing has been measured by dozens of different measures, which have many different rating scales. I'm using the following equation to transform scores between two different measures to put them on the same metric:

$$X_2 = \frac{(X_1 - \min_1)(\max_2 - \min_2)}{\max_1 - \min_1}+\min_2$$

Where $X_2$ = the score on the second measure; $X_1$ = score on the first measure I want to transform; $\min_1$ and $\max_1$ = lowest and highest possible scores on measure 1; and $\min_2$ and $\max_2$ = lowest and highest possible scores on measure 2. (Card, N., Applied Meta-Analysis for Social Science Research, 2011, pg. 148).

Assume that any two measures I'm trying to compare are assessing the same construct. I know I can use this if the two measures have the same rating scale (1–5, strongly agree to strongly disagree). My questions are:

  1. What about two measures that have the same number of response options, but slightly different wording? For example, both are 5 point scales, but measure 1 is:

    1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree

    while measure 2 is:

    1 = completely disagree, 2 = disagree up to a point, 3 = neutral, 4 = agree up to a point,
    5 = agree completely

    How critical is it that they have exactly the same wording? (I'm inclined to think these are roughly the same and could be combined).

  2. What about measures with different numbers of responses? Ex: measure 1 is the same as the example above, but measure 2 has a 1–7 rating scale with only endpoint anchors of strongly disagree/strongly agree, measure 3 has a 1–6 rating scale of:

    1 = strongly disagree, 2 = moderately disagree, 3 = slightly disagree, 4 = slightly agree, 5 = moderately agree, 6 = strongly agree

    and measure 4 has a 1–7 rating scale of:

    1 = strongly agree, 2 = agree, 3 = slightly agree, 4 = neutral, 5 = slightly disagree, 6 = disagree, 7 = strongly disagree

    Converting between 5 and 7 point scales seems less problematic than converting between 5 and 6, since you lose the neutral midpoint. However, I welcome any advice on this as it will inform my methods moving forward. Thank you!

$\endgroup$
  • $\begingroup$ This is going to be a tough one, because Likert scales don't produce the right kind of data. You'd want interval data for your equation, because you'd want to be able to treat responses that are the same scaled distances from their respective minima as equal. Ordinal data are not so nice as this. Consider a related problem, "Is 0 a valid value in a Likert scale?" If you had all the original data and not just scores, you might be able to use a partial credit model; otherwise this is probably doomed to be an inexact comparison (more so, that is). $\endgroup$ – Nick Stauner May 20 '14 at 0:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.