Let's say I have some self-report items measured on a 5-point Likert scale (Strongly Disagree to Strongly Agree) and other items measured on a 4-point Likert scale (Never, Rarely, Sometimes, Often). Can anyone point me to literature (or practical advice) on combining these items into a composite scale? Let's assume for the sake of argument that we have some empirical evidence that items should be combined.
1. Sum raw scores
Con: Max responses on the 5-point scale (originally 4's) drive up the total scale score more than max responses on the 4-point scale (originally 3's).
2. Rescale and sum
Put all of the items on a 0-1 scale and sum. So the 4-point items (0,1,2,3) would be multiplied by (4/3)/4 and the 5-point items (0,1,2,3,4) would be multiplied by 1/4, resulting in possible values of (0,.33,.66,1) and (0,.25,.50,.75,1), respectively. This way max responses on the 5-point scale (originally 4's) would not drive up the total scale score more than max responses on the 4-point scale (originally 3's).
Pro: Items would have equal weighting. (could be a con, depending on your perspective).
Con: Ignores differences in variability between items on different metrics?
3. Standardize and sum
A related approach would be to standardize all of the items (z score) and then sum.
Pro: Addresses differences in variability between items on different metrics
Con: Total scale score becomes less interpretable and sample-specific. The latter makes it hard to benchmark as a measure to be used in other settings/other samples.
4. PCA or other data reduction
4a. EFA to get factor loadings. Multiply scaled items by factor loadings.
4b. PCA to get score of first principal component.
Pro: Items weighted by influence.
Con: Same as #2. EFA-derived scores could vary a lot depending on rotation/extraction choices. Some would not advise on ordinal data.
Overall: I like #2 because it seems easier to compare results across different samples. Thoughts? Alternative ideas or concerns about the ideas presented?