How does one run a meta-analysis on indirect / mediated effects?

I have two different datasets, and am reporting the same mediation analysis in both. The mediation in one study is clear, in the other less so. I am trying to run a small meta-analysis across both mediations to see if there is an overall effect. Using Preacher and Kelley (2011), I have derived a kappa-squared for each mediation.

I'm used to working with r and d-statistics in my meta analyses. Is there a method of transforming kappa-squareds into r-statistics? Conversely, is there a program that will allow me to run a meta analysis using only the kappa-squareds?

• Hi Ann: Can you clarify which path(s) in your mediation model you are trying to meta-analyze? Is it the path from X-->Y (the "c" path, or total effect); X-->M (the "a" path); M-->Y (the "b" path); X-->Y after controlling for M (the "c' " path, or the direct effect), or the estimate of the indirect/mediated effect (the "ab" path)? I think I can provide an answer (which actually involves meta-analyzing something other than kappa^2), but it would be best to know which path you want to meta-analyze... Dec 20 '15 at 6:38
• Hi jsakaluk, thank you! We're interested in the estimate of the indirect/mediated effect (the "ab" path). Dec 23 '15 at 1:24

I am not sure if the kappa-squared is a good measure of mediation effect size (see Wen & Fan, 2015). Since there are only two studies, there is no need to apply meta-analysis.

If the measures are the same in these data sets (the two assumptions listed by jsakaluk), we may test the equality of the unstandardized indirect effects by imposing a nonlinear constraint $H_0: \alpha^{(1)}\beta^{(1)}=\alpha^{(2)}\beta^{(2)}$, where $\alpha$ and $\beta$ are the population parameters of the paths $a$ and $b$, and the superscript represents the group. The models with and without this constraint are nested. A likelihood ratio test with 1 df may be used to test this null hypothesis. If your SEM program does not allow nonlinear constraint, we may compute a function of the parameters, say $p=a^{(1)}b^{(1)}-a^{(2)}b^{(2)}$. A Wald test can be used to test whether the population value of p is 0 (see Cheung, 2007).

When there are many groups or studies, meta-analytic methods are better than the multiple-group SEM to synthesize indirect effect. Cheung and Cheung (in press) consider two approaches. The first approach is applying the meta-analytic structural equation modeling (MASEM) to estimate a pooled correlation matrix with a random-effects multivariate meta-analysis. A mediation model is then fit on this pooled correlation matrix in the stage two analysis. This approach can only test the "average" indirect effect. However, it does not test whether the indirect effects are homogeneous.

The second approach is to calculate the standardized indirect effect (and possibly the standardized direct effect) in each study. These effect sizes are meta-analyzed with a multivariate meta-analysis. The main advantage of this approach is that it quantify the heterogeneity of the indirect and direct effects. Cheung and Cheung (in press) discuss the advantages and limitations of these two approaches. One cautionary note on calculating the sampling variance/covariance of the standardized indirect effect in the second approach is that the standard errors calculated by standardizing the variables are likely incorrect (Cheung, 2009; Yuan & Chan, 2011). It is preferable to use the indirectEffect function in the metaSEM package implemented in R to calculate the correct standard errors for the standardized indirect and direct effects.

References

Cheung, M. W.-L. (2007). Comparison of approaches to constructing confidence intervals for mediating effects using structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 14(2), 227–246. http://doi.org/10.1080/10705510709336745

Cheung, M. W.-L. (2009). Comparison of methods for constructing confidence intervals of standardized indirect effects. Behavior Research Methods, 41(2), 425–438. http://doi.org/10.3758/BRM.41.2.425

Cheung, M. W.-L., & Cheung, S. F. (in press). Random-effects models for meta-analytic structural equation modeling: Review, issues, and illustrations. Research Synthesis Methods.

Wen, Z., & Fan, X. (2015). Monotonicity of effect sizes: Questioning kappa-squared as mediation effect size measure. Psychological Methods, 20(2), 193–203. http://doi.org/10.1037/met0000029

Yuan, K.-H., & Chan, W. (2011). Biases and standard errors of standardized regression coefficients. Psychometrika, 76(4), 670–690. http://doi.org/10.1007/s11336-011-9224-6

So here's an option to meta-analyze your indirect/mediated effects that does not use $\kappa^2$ (because I don't know how to calculate a variance/standard error for $\kappa^2$).

In order for this method to be valid, you must:

1. Be running the exact same mediation model in both datasets (i.e., same IV, same M, same DV, and same covariates, if any).
2. Be using the exact same measures of variables in both datasets, scaled the same way.*

If you meet those two criteria, then the actual execution of your small meta-analysis is easy. What you need to retrieve are the unstandardized estimates of the ab, and the estimated standard error for each ab path (this should be provided with most mediation-assessing tools, e.g., Hayes's PROCESS macro for SAS and SPSS). Unstandardized regression weights are effect sizes (albeit unstandardized ones), so as long as you have their variance or standard error, you can meta-analyze them as long as the regression weights come from comparable models (criteria 1 and 2 above).

From that point, you can just plug the estimates and standard errors into a spreadsheet, and use your favourite meta-analysis software (most will accept standard errors--I like metafor for R), or calculate your fixed effects model by hand by squaring the standard errors to variances, and using the inverse-variance weighted approach.

*You could, alternatively, standardize all variables for both models pre-analysis, and then do the same thing I describe, if it's important for you to meta-analyze a standardized effect size.