Statistical criteria (correlation) for aggregating two items together in psychology? I have a question regarding item aggregation. I have conducted a study where I measured 4 items. Two of these items are conceptually linked together and seem to measure the same dimension, and the other two are also conceptually linked and I would also say they measure a similar dimension.
However, when I check correlation two by two for these items, they are only around 0.4 (in both cases).
I wondered if there existed any articles or references that talked about this issue or gave recommendations on what is an appropriate correlation strength in order to aggregate two items together.
Thank you in advance for your answer !
 A: You could look into innumerable sources on scale development, psychometrics, validity (including construct validity) and reliability.  There are websites, books, and articles galore.  You could search for posts on this site.
But no matter what you might find is commonly done, will you feel you have made adequate use of your data (which I presume took some time, effort, and resources to generate and collect) if you combine, by averaging or otherwise, scores that explain only about 1/6 of the variance in each other?  As for the remaining 5/6 of the information, you will essentially be throwing it away.
No matter what your motivation for combining, your results so far tell you that these items are largely measuring different things, supplying different types of information.  Without a strong reason otherwise, you should honor those different "roles" they play by including the items as separate variables in your main analysis.  That is, if you include the items at all.  Perhaps you will determine, from other investigation, that one or more of them is not a sufficiently valid or reliable indicator of any construct relevant to your research.  But then you would be throwing its information away for a sound reason.
