5
$\begingroup$

A colleague wants to analyze two outcomes (X1, X2). They believe the two outcomes measure the same construct or something similar. They decide to Z-score both outcomes in separate dataset and combine them (X3). They assume that because they are Z-scored they are now on the same scale, and because they are both measuring the same outcome, they can be combined.

A real-world example could be that a person has depressive symptoms measure by the Center for Epidemiological Studies - Depression Scale and the Beck Depression Inventory, in two different datasets and in two different scales. So, the “construct” is the same but the scales and measurement are different. They want to combine these, but the measures are on different scales. They decide to Z-score and concatenate. But this is inappropriate, no?

In other words, X1 exists only in dataframe 1 and X2 exists only in dataframe 2. The mean, principal component or latent factor cannot be estimated as they are variables in separate datasets. The person just Z-scores and concatenation the variables into X3 to assume they are the same variable.

I am aware that simply standardizing scores does not necessarily make variables interchangeable or equivalent. However, I am unclear if there is any simulation-based evidence or literature that can make this point clear. I want to make clear that two variables that you assume to measure the same construct on different metrics/scales cannot simply be combined because they have been Z-scored. What is the reason for this? What is a good way to communicate this point that X1 and X2 both being Z-scored does not mean they can be combined?

$\endgroup$

2 Answers 2

2
$\begingroup$

The idea behind the standardization seems to be 'if we do this, they are on the same numeric range, so then they mean the same'

  • If it is about having the same numeric range, it makes more sense to use something like $\frac{y - Min_{scale}}{Max_{scale} - Min_{scale}}$, then both scales range from 0 to 1 (although the observed ranges may still be different of course)

  • Either of these approaches assumes that the functional form describing the relation between the scale / measurement and the construct of interest is the same for both scales (up to the transformation). That seems very unlikely

    • Take 'size of a cube' as construct of interest and the length ($l$) and the volume ($V$) as the two scales measuring it.
    • It should be clear that using a calibration formule $V = l^3$ would allow to translate and combine some volume-measures with some length-measures (either by translating volume into length or the other way around)
    • The graph below shows the distributions of both scales after standardization in an identical population of cubes (simulated cubes and expressed size in both scales for all cubes). As they are really different (standardized volume more skewed), it should be clear that they cannot reasonably combined that way

enter image description here

  • To me, the most important reason why you should see this cannot work, is that the distribution of the observed scale (and hence the standardisation) is as much a property of the sample as it is from the scale itself. This means it doesn't make sense to standardize and combine even if in both cases the scales / instruments to measure are identical! If you restrict some physical test (which would represent the scale here) to 90-95 year olds, the distribution will be very different from when you apply the same scale to 20-50 year olds. Standardizing these would mean that after standardization 'ran 100 meters in 75 seconds' from the first group would get the same numerical value as 'ran 100 meters in 18 seconds' in the second group. If combining identical scales using standardization makes no sense, then combining different scales doesn't make sense either
$\endgroup$
1
  • $\begingroup$ Great answer, thank you! $\endgroup$
    – JElder
    Nov 11, 2023 at 12:24
6
$\begingroup$

My quick answer is: if the properties of the two scales are different, then combining the two scales by z-scoring will likely introduce hidden errors into any subsequent analysis. For example,

  • if one scale is skewed and another symmetric, z-scoring will still result in one scale being skewed and the other symmetric
  • if one scale favours certain conditions and the other favours different kinds of conditions
  • etc.

then z-scaling will not magically make this disappear and make them equivalent! I'm not saying that it's always inappropriate, but you should carefully consider the properties of the two scales being combined (from both a theoretical, and an empirical perspective) before making this decision.


In R, a quick example:

  • I'm generating two set of scores, one Normal and the other right-skewed
  • I'm then z-scaling the scores separately, and then plotting them

Would I combine these two z-scores? Probably not at this stage, since the way that they are distributed post-transformation is still very different.

set.seed(123)

scores_group_1 <- rnorm(n = 100, mean = 50, sd = 20)
scores_group_2 <- rgamma(n = 100, shape = 1, scale = 10)

z_group_1 <- scale(scores_group_1, center = TRUE, scale = TRUE)
z_group_2 <- scale(scores_group_2, center = TRUE, scale = TRUE)

par(mfrow = c(2,1))
hist(z_group_1, breaks = seq(-4.75, 4.75, by = 0.5))
hist(z_group_2, breaks = seq(-4.75, 4.75, by = 0.5))
par(mfrow = c(1,1))

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.