3
$\begingroup$

I have a quick question about the metafor package. I have been using the author's guide for stochastically dependent effect sizes (via github) to run a [multivariate meta-analysis][1] using multiple variables from the same treatment studies.

We have a special interest in looking at whether one variable decreases as the other(s) decrease. So we are very interested in the correlational output that metafor also supplies, while also controlling for some of the shared variance.

My question is how do you interpret the rho part of the output? Is this a simply a pearson correlation that attempts to estimate Rho at the population level or is this a rank-ordered Spearman's rho correlation? I need to test the significance of those correlations, hence I need to know how to interpret the rho columns.

This is the example output from the website above after running a 2-outcome multivariate meta-analysis:

         rho.math  rho.vrbl    math  vrbl 
math           1    -1.000       -    no 
verbal    -1.000         1       5     -

An example of my output is:

                rho.a___  rho.d___  rho.e___  rho.em__  rho.i___    a___  d___  e___  em__  i___ 
a______                1     0.825     0.541     0.878     0.717       -    no    no    no    no 
d_________         0.825         1     0.921     0.935     0.822      57     -    no    no    no 
e___________       0.541     0.921         1     0.783     0.741      39    47     -    no    no 
em__________       0.878     0.935     0.783         1     0.611      10    10     4     -    no 
i_____________     0.717     0.822     0.741     0.611         1      37    45    22     2     -

Any thoughts or help would be appreciated. Thanks!

[1]: https://wviechtb.github.io/meta_analysis_books/cooper2019.html#13)_Stochastically_Dependent_Effect_Sizes

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

The values in the table are the estimated correlations between the random effects, which are assumed to follow a multivariate normal distribution. So these are the ML/REML estimates of the correlations in that multivariate normal distribution.

How to call them in a matter of personal preference. I wouldn't call them 'Pearson correlations' because that might suggest that they are obtained by taking two observed variables and computing the Pearson product-moment correlation coefficient based on them, but that is not how they are obtained. They are definitely not 'rank-order' correlations.

If you need to test them, you cannot use standard methods that are used for testing Pearson product-moment correlation coefficients. Instead, you can do likelihood ratio tests, where you compare the fitted model with a model where you constrain one (or multiple) of the correlations to 0 (you can do this via the rho argument; see help(rma.mv) and especially the section on "Fixing Variance Components and/or Correlations").

Improve this answer
$\endgroup$
4
  • $\begingroup$ Oh boy, I've been interpreting this wrong then, but thanks for the helpful explanation. What if I don't necessarily want to test the correlations, but want to compare their magnitude across models. . e.g. I ran an overall multivariate meta-analysis model which is actually the output above (just the correlation part). Now say I've run sensitivity analyses for the meta-analysis - all of models produce different Rho values as per the criteria I've selected for sensitivity. . Can I compare the correlations produced by the overall model with correlations in the sensitivity models some how? $\endgroup$
    – alessothegreat
    Commented Jun 25, 2021 at 12:12
  • $\begingroup$ Or could I just show the pattern of Rho results without any tests and explain that Rho = .90 (e.g.) suggests a stronger estimated random effects correlation between the two variables than Rho = .70 (e.g.). We are interested in which variables selected for sensitivity analyses improve the correlations from the overall model. $\endgroup$
    – alessothegreat
    Commented Jun 25, 2021 at 12:28
  • 1
    $\begingroup$ Testing them against each other would be difficult since they are based on the same data (or similar data depending on how your sensitivity analyses are set up) and hence the correlations across your analyses are not independent. One possibility would be some kind of bootstrapping approach. Of course you can always just report the estimates and state which one was larger, but this is not proper evidence that there is actually a significant difference between them. $\endgroup$
    – Wolfgang
    Commented Jun 27, 2021 at 11:46
  • $\begingroup$ Thank you - that helps to confirm what can be said at this time. $\endgroup$
    – alessothegreat
    Commented Jun 27, 2021 at 14:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.