A colleague of mine wishes to conduct a network meta-analysis of correlation coefficients for a set of imaging tests (eg ultrasound, computed tomography, magnetic resonance imaging, and so forth).

He has collected correlation coefficients (R), R-squared, and Fisher z. He pushes me to embark in a formal network meta-analysis (eg a frequentist one with the netmeta R package).

However, after some thoughts, I think it is a logical paradox, in the sense that in my opinion the transitivity assumption (see figure below) does not apply to correlation coefficients in the same fashion it applies to clinical trials with unambiguous endpoint definitions or to diagnostic test accuracy studies with clear labeling of healthy and diseased subjects.graphical synthesis of the transitivity assumption

Indeed, I fear that the correlation obtained when comparing test A and B, and that obtained when comparing B and C, cannot inform on the comparison between A and C, as the latter may depend on completely different cases.

I have also searched in Google and PubMed for "network meta-analysis" and "correlation", but did not find any meaningful reference.

According, my recommendation would simply to stick to a univariate meta-analysis approach, or otherwise use a multivariate meta-analysis one, for instance with mvmeta in Stata or R.

Am I correct?

  • 1
    $\begingroup$ See: Salanti, G. (2012). Indirect and mixed‐treatment comparison, network, or multiple‐treatments meta‐analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Research synthesis methods, 3(2), 80-97. This addresses the issue you raise. $\endgroup$ – dbwilson Feb 21 '19 at 14:23
  • $\begingroup$ @dbwilson I checked Salanti's paper but I think it is not really pertinent as it does not refer explicitly to correlation coefficient or Fisher z. $\endgroup$ – Joe_74 Feb 21 '19 at 16:02
  • 1
    $\begingroup$ It doesn't matter what type of effect size is being used. The logic and math are the same -- you have an effect size (Fischer's Zr) and its associated inverse variance weight. From that point forward, meta-analysis, including network meta-analysis proceeds in the same way. As such, the issues related to direct and indirect effects and the various assumptions being made are the same. $\endgroup$ – dbwilson Feb 21 '19 at 19:47
  • $\begingroup$ #dbwilson I understand your point in general, and indeed have performed NMAs of RCTs and DTA studies. However, there you have clear and universally unequivocal cases. In correlation this is not true, in the sense that a given subject may lead correlation between A and B, whereas another subject may lead correlation between B and C. Thus, you end up inferring inappropriately on the correlation between A and C. $\endgroup$ – Joe_74 Feb 22 '19 at 9:00

There are two points to comment on. Note that correlation coefficients do not measure differences, but agreement.

It is also right that correlation coefficients per se are not transitive. Of course it is possible that cor(x,y) > 0, cor(y,z) > 0, but cor(x,z) < 0. Set, for example, in R

x <- c(1,2,1,3,2,1,3,2,3,1,2,3); y <- c(0,1,1,2,2,2,3,3,4,3,4,5); z <- c(2,1,3,0,2,4,1,3,2,5,4,3); cor(x,y); cor(y,z); cov(x,z);

Most important, we have to distinguish between meta-analysis of one-sample measures and meta-analysis of two-sample comparisons.

A pairwise meta-analysis of treatments effects (using effect measures such as the mean difference, the relative risk, or the odds ratio) involves the comparison of two groups, and therefore the natural extension is network meta-analysis involving more than two groups. Traditionally, this has been based on a contrast-based approach. The consistency assumption says that the direct evidence and the indirect evidence agree, formally:

d(A-C) = d(A-B) + d(B-C)

By contrast to this, meta-analysis of one-sample measures (such as means, proportions, or incidence rates, or correlation coefficients) does not involve a comparison of two groups. I do not see any meaningful extension of one-sample meta-analysis to network meta-analysis.

A different thing would it be to compare correlation coefficients of two or more different studies by looking at their (transformed) differences, in analogy to comparing means, proportions or incidence rates. For such differences, a consistency assumption could be made (though I doubt that this would make much sense).

Gerta Rücker


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.