1
$\begingroup$

I already asked about this in stackoverflow but maybe it fits better here.

I want to conduct a multilevel meta-analysis using metafor in R. I have effect sizes ("esid") nested within samples ("sampleid") nested within publications ("studyid"). I have four subgroups ("task.type").

The mean effect sizes for each subgroup differ depending on whether I use task.type as a moderator or run the rma.mv command for each subgroup independently.

This is the code I use with task.type as moderator:

multi.task <- rma.mv(yi=g, V = var.g, data=df, random=list(~ 1 | esid, ~1 | sampleid, ~1 | studyid), mods=~factor(task.type)-1)

This is the one I use when computing the model for each task.type individually:

task.X <- rma.mv(yi=g, V = var.g, data=df, subset=(task=="X"), random=list(~ 1 | esid, ~ 1 |sampleid, ~ 1 | studyid))

Means for the four task types using the first method (task.type as moderator) vs the second method (rma.mv for each task.type):

  1. 0,61 vs -0,06
  2. -0,52 vs 0,33
  3. 0,58 vs 0,39
  4. 0,62 vs 0,61

Why are the results so different?

$\endgroup$
3
  • $\begingroup$ See metafor-project.org/doku.php/… and ask again if that does not answer your question. $\endgroup$
    – mdewey
    Apr 15, 2018 at 12:47
  • $\begingroup$ As far as I understand, this does not explain the huge differences I find for the mean effect sizes of my subgroups. Also, I am already using the rma.mv command. The problem does not occur if I use the rma command (which is not suitable for my hierachical data, though). $\endgroup$
    – Max J.
    Apr 16, 2018 at 20:01
  • 1
    $\begingroup$ This might get better responses on the R meta-analysis mailing list stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis/ $\endgroup$
    – mdewey
    Aug 12, 2018 at 13:02

2 Answers 2

1
$\begingroup$

I think it is because when you fit separate random-effects meta-analysis models on the individual task type subsets (your second method), you allow the amount of heterogeneity within each set to be different.

In contrast, when you fit a single meta-analysis model and include task.type as a moderator (your first method), there is a single variance component for the amount of residual heterogeneity, which implies that the amount of heterogeneity within each subset is assumed to be the same.

This in turn affects the means given by the model (if your task type subsets each have different amounts of heterogeneity) and is explained by Wolfgang here.

$\endgroup$
0
$\begingroup$

You can find answer in Conducting Meta-Analyses in R with the metafor Package page 18

for subset: which illustrates the use of the subset argument (which can either be a logical vector as used here or a numeric vector indicating the indices of the observations to include). However, unless differences in the amount of heterogeneity are of interest or suspected to be present within the different levels, this is not an ideal approach

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.