7
$\begingroup$

I'm in the process of performing an Structural Equation Modelling (SEM) meta-analysis on some psychological data. Ultimately I want to examine a mediational model based on a set of correlation matrices. I've spent a little time reviewing the literature regarding what to do when you have heterogenous correlation matrices. I've written up a brief review of what I've found on Cognitive Sciences SE.

To summarise what I've found:

  • Most researchers use a two-step approach. Step 1 involves extracting estimates of true correlations between a set of variables using a variety of meta-analytic techniques. Step 2 involves inputting the correlation into your SEM software and in general doing analyses as per normal non-meta-analytic SEM.
  • Despite an awareness of the problems, many researchers still use the two-step approach even though most datasets have variation in true correlations that can not merely be attributable to random sampling (i.e., a random effects model is generally more applicable).
  • Some researchers mention the possibility of clustering the dataset or performing SEM Meta-analysis on values of moderators in the hope that the true score variability will be removed once you divide the analysis up into groups. However, in most applications such moderators are insufficient to explain all the true score variation.

So instead of adopting a two-step approach, I thought about maybe doing the following, at least for cases where you are able to compute a full correlation matrix for each study:

"perform SEM on each sample and treat the parameters and fit statistics as values that vary between samples. You could then summarise the distribution (e.g.,mean and SD) of these SEM parameters and fit statistics. This would be similar to how correlations and other effect sizes are typically modelled as random effects. So, for example you could look at the variation in the indirect effect across samples. The challenging bit would be to tease out what is true score variation and what is due to random sampling."

Excluding the issue of teasing out true score from random sampling, it would be straight forward to do such an analysis in R with any of the SEM packages.

Questions

So, I was wondering:

  • Has this approach of fitting SEM to each study separately ever been applied?
  • What are the pros and cons of the approach?
  • How might you tease out what variation in SEM parameters is due to random sampling and what is due to true score variance?
  • Or is there some better approach to doing SEM meta-analysis on heterogenous correlation matrices?
$\endgroup$
5
$\begingroup$

Cheung and Cheung (2010) gave a presentation discussing two appraoches to meta-analytic Structural Equation Modelling (SEM). They label the two approaches:

  1. Correlation matrices Meta-Analysis: An average correlation matrix is extracted from the primary studies and SEM is applied to this single correlation matrix.
  2. Model Parameters Meta-Analysis: Model fitting is applied separately to the correlation matrix of each primary study and the variation in parameters is modelled using random effects multivariate meta-analysis (Cheung and Cheung, 2010 cite Cheung 2009, and Kalaian & Raudenbush, 1996)

Cheung and Cheung (2010) note that this is an area of research needing further development. They recommend trying both approaches especially where the full correlation matrix is available at least in a subset of studies.

Discussions of how to perform meta-analysis SEM are also closely related with general discussions of multivariate meta-analysis (e.g., see the tutorial by Van Houwelingen et al 2002).

I think that the standIndirect function in the metaSEM package in R (Cheung, 2011) implements a three variable test of mediation using the model parameters meta-analysis approach.

References

  • Becker, B.J. & Wu, M.J. (2007). The synthesis of regression slopes in meta-analysis. Statistical Science, 22, 414-429. PDF on arxiv
  • Cheung, S. F., & Cheung, M. W.-L. (2010). Random effects models for meta-analytic structural equation modeling. Presented at The 7th Conference of the International Test Commission, July 19-21, 2010, Hong Kong.
  • Cheung, M.W.L. (2011). metaSEM: Meta-Analysis using Structural Equation Modeling.
  • Cheung, M.W.L. (2009). Comparison of methods for constructing confidence intervals of standardized indirect effects. Behavior Research Methods, 41, 425-438.
  • Cheung, M.W.L. (2009). Constructing approximate confidence intervals for parameters with structural equation models. Structural Equation Modeling, 16, 267-294.
  • Kalaian, H.A. & Raudenbush, S.W. (1996). A multivariate mixed linear model for meta-analysis.. Psychological methods, 1, 227. PDF
  • Van Houwelingen, H.C., Arends, L.R. & Stijnen, T. (2002). Advanced methods in meta-analysis: multivariate approach and meta-regression. Statistics in medicine, 21, 589-624. PDF
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.