3
$\begingroup$

I am using a pre-existing universal tested scale to measure a specific construct. The construct is measured through 5 items on a 5 points likert scale. However, I would probably need to change the range of all the items scale from 5 to 7. Would this affect in any way the validity and reliability of the construct?

In fact, I keep seeing researchers changing the points range of items for their own purposes. However, I am not sure whether this would affect in any way the reliability or validity of the scale. Indeed I have been told that this practice is wrong and would negatively affect my analysis.

Furthermore, sometimes I see researchers using even smaller version of the same scale with less items. If that is fine I could possibly changing the point range affect validity and reliability of the scale?

$\endgroup$
1
$\begingroup$

Yes, the number of questions and the number of response categories can change the measurement properties of the scale. Any change to a scale should be evaluated.

Reducing the number of categories might result in a loss of information and increasing the number of categories does not necessarily increases the information or precision of a scale.

Furthermore, a reduction of items is sometimes possible without major loss of information, but again this is something that should be evaluated.

Item response theory (IRT) provides the framework to investigate these issues with scales.

$\endgroup$
0
$\begingroup$

Setting apart subtle issue with face validity (or concept close to the coverage and acceptability of the questionnaire), changing the number of response options will likely affect the variance of the response pattern matrix, hence any estimate for the internal consistency of the scale (Cronbach alpha or Guttman $\lambda_6$), especially when they are calculated on item variances and total test variance, i.e. using the covariance matrix. Here is a toy example using the R software, where I recode a subscale of a personality inventory originally scored on a 6-point scale to a 3-point scale:

library(psych)

## personality test: Agreeableness, Conscientiousness,
## Extraversion, Neuroticism, and Opennness.
## 25 items (5 factors) scored on a 6-point response scale:
## 1 Very Inaccurate 2 Moderately Inaccurate 3 Slightly Inaccurate
## 4 Slightly Accurate 5 Moderately Accurate 6 Very Accurate.
data(bfi)

d <- bfi[,1:5]
apply(d, 2, var, na.rm = TRUE)
#       A1       A2       A3       A4       A5
# 1.981724 1.373631 1.694771 2.189313 1.583853
alpha(d, check.keys = TRUE)
# 0.70 95% CI [0.69; 0.72]

recode <- function(x) {
  x[x == 1] <- 2
  x[x == 4] <- 3
  x[x == 5 | x == 6] <- 4
  return(x-1)
}

d2 <- as.data.frame(lapply(d, recode))
apply(d2, 2, var, na.rm = TRUE)
#        A1        A2        A3        A4        A5
# 0.4690391 0.3601790 0.4379549 0.4969999 0.4260348
alpha(d2, check.keys = TRUE)
# 0.66 [0.64; 0.68]

So, the Cronbach alpha drops down just because I (artificially) reduced the variance of item and total score.

Moreover, although there are lot of debates about the optimal number of response options for Likert-type items (5 to 7 appear to be the most frequently solution), the problem is not so much how it affects the reliability of scale scores or internal consistency as how it affects the problems of transition from one response modality to another, in probabilistic terms, which are known as threshold reversals in, e.g., the Partial Credit Model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.