I am currently working on a meta-analysis, for which I need to analyze multiple effect sizes nested within samples. I am partial to Cheung's (2014) three-level meta-analysis approach to meta-analyzing dependent effect sizes, as opposed to some of the other possible strategies (e.g., ignoring dependency, averaging effect sizes within studies, selecting one effect size, or shifting the unit of analysis). Many of my dependent effect sizes are correlations involve fairly distinctive (but topically related) variables, so averaging across them does not make conceptual sense, and even if it did, it would cut my number of total effect sizes to analyze by nearly half.
At the same time, however, I am also interested in using Stanley & Doucouliagos's (2014) method of addressing publication bias in the course of estimating a meta-analytic effect. In a nutshell, one either fits a meta-regression model predicting study effect sizes by their respective variances (the precision effect test, or PET), or their respective standard errors (the precision effect estimate with standard errors, or PEESE). Depending on the significance of the intercept in the PET model, one either uses the intercept from the PET model (if the PET intercept p > .05) or the PEESE model (if the PET intercept p < .05) as the estimated publication-bias-free mean effect size.
My problem, however, stems from this excerpt of Stanley & Doucouliagos (2014):
In our simulations, excess unexplained heterogeneity is always included; thus, by conventional practice, REE [random-effects estimators] should be preferred over FEE [fixed-effects estimators]. However, conventional practice is wrong when there is publication selection. With selection for statistical significance, REE is always more biased than FEE (Table 3). This predictable inferiority is due to the fact that REE is itself a weighted average of the simple mean, which has the largest publication bias, and FEE.
This passage leads me to believe that I should not be using PET-PEESE in random-effects/mixed-effects meta-analytic models, but a multilevel meta-analytic model would seem to require a random-effects estimator.
I am torn as to what to do. I want to be able to model all of my dependent effect sizes, but simultaneously take advantage of this particular method of correcting for publication bias. Is there some way for me to legitimately integrate the 3-level meta-analysis strategy with PET-PEESE?
Cheung, M. W. L. (2014). Modeling dependent effect sizes with three-level meta-analyses: A structural equation modeling approach. Psychological Methods, 19, 211-229.
Stanley, T. D., & Doucouliagos, H. (2014). Meta-regression approximations to reduce publication selection bias. Research Synthesis Methods, 5, 60-78.