A colleague wants to analyze two outcomes (X1, X2). They believe the two outcomes measure the same construct or something similar. They decide to Z-score both outcomes in separate dataset and combine them (X3). They assume that because they are Z-scored they are now on the same scale, and because they are both measuring the same outcome, they can be combined.
A real-world example could be that a person has depressive symptoms measure by the Center for Epidemiological Studies - Depression Scale and the Beck Depression Inventory, in two different datasets and in two different scales. So, the “construct” is the same but the scales and measurement are different. They want to combine these, but the measures are on different scales. They decide to Z-score and concatenate. But this is inappropriate, no?
In other words, X1 exists only in dataframe 1 and X2 exists only in dataframe 2. The mean, principal component or latent factor cannot be estimated as they are variables in separate datasets. The person just Z-scores and concatenation the variables into X3 to assume they are the same variable.
I am aware that simply standardizing scores does not necessarily make variables interchangeable or equivalent. However, I am unclear if there is any simulation-based evidence or literature that can make this point clear. I want to make clear that two variables that you assume to measure the same construct on different metrics/scales cannot simply be combined because they have been Z-scored. What is the reason for this? What is a good way to communicate this point that X1 and X2 both being Z-scored does not mean they can be combined?