I'm curious if anyone knows what it would be called if you correlated the results across two studies.

For example: Two published studies looked at the correlation between an identical set of DVs (say a set of questionnaires), but two different IVs (like two different disease diagnoses). Importantly, the underlying data is not accessible, and the participants used for the two studies may be completely different people, or they may overlap partially (or even fully). One way of testing how similar the results of the two studies are would be to correlate the results (a correlation of two vectors, each consisting of effect-sizes from the corresponding study). A high correlation would indicate that the two DVs show a similar pattern of associations with the IVs.

My question is, is there a general name for this kind of analysis?

In population genetics, it would be called a 'cross-trait ld-score regression' (see this article). But what if I wanted to do something similar, but with questionnaire data, as in my example? What would it be called, and what would the exact procedure be for conducting it? For instance, how do you handle correlated IVs, are there known procedures for adjusting for sample overlap, what if you want to include multiple questionnaires of different lengths?

EDIT: The sort of analysis I describe has also been done recently using neuroimaging data; see this paper. A series of prior studies each examined the association of a psychiatric diagnosis with brain structure, brain structure being defined identically across all studies. For the correlation between studies, brain regions are the rows, each study is a column, and the value of each cell is the effect-size of the association between a diagnosis (column) with a measurement of the brain (row). The correlation between results is simply the correlation between the columns. Unfortunately, the method is not named. I also have a range of concerns about the method, and am hoping that other people who know more about this sort of thing than I may have already given some thought to how to appropriately do this kind of analysis.

  • 1
    $\begingroup$ Exactly how are you matching "correlations from the corresponding study" in order to compare the two studies? $\endgroup$
    – whuber
    Jun 11, 2020 at 17:50
  • $\begingroup$ The correlations are to the same set of IVs. Say for instance, different personality traits. The same set of variables, measured identically, are present in both studies. The one difference is the DV. The results from each study are then ordered identically, and then correlated. $\endgroup$
    – David B
    Jun 11, 2020 at 18:04


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.