2
$\begingroup$

I'm wondering why I get slightly different estimates for a meta-analyzed set of effect sizes, when using the rma.mv function from the metafor package, compared to the meta3 function from the metaSEM package.

Here's a reproducible example:

#Install and call both packages
install.packages("metafor")
install.packages("metaSEM")
library(metafor)
library(metaSEM)

#Use example dataset from metafor package
dat <- get(data(dat.konstantopoulos2011))

#Fit multilevel meta model, with effect sizes nested in district
#Use maximum likelihood estimator (default is REML, but meta3 uses ML)
res <- rma.mv(yi, vi, random = ~ 1 | district, data=dat, method = "ML")
summary(res)

#Fit multilevel meta model, with effect sizes nested in district
res.alt = meta3(y = yi, v = vi, cluster = district, data=dat)
summary(res.alt)

Both approaches should be able to account for dependency between effect sizes in the dataset--in this example, they are standardized mean differences nested within school district.

But while I understand that rma.mv fits a 2-level model, and meta3 fits a three level model, I don't understand why the estimated mean effect size differs between the two approaches. Shouldn't the 3-level model just further partition the variance component from the 2-level model into an additional source? Why is the estimated mean also impacted?

At a more applied level, I suppose I am also wondering: how does one choose between the two approaches, when wanting to meta-analyze nested effect sizes?

$\endgroup$
4
$\begingroup$

You are not fitting the same models. Please read http://www.metafor-project.org/doku.php/analyses:konstantopoulos2011, which explains in detail how to fit the three-level model with rma.mv(). In particular, you should use:

res <- rma.mv(yi, vi, random = ~ 1 | district/school, data=dat, method = "ML")

Results are the same then.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.