3
$\begingroup$

I am using the metafor package to assess publication bias for a meta-analysis (included approx 100 and approx 900 individual effect sizes). I am facing an issue with the implementation of the trim and fill procedure:

  1. I have produced the funnel plot, which visually provides indication for some missing studies on the left side.

  2. I have then conducted the Egger's test and find a significant p-value, thus confirming plot asymmetrie.

  3. Then I have run the trim and fill procedure (using the trimfill command of the metafor package), but oddly, no missing studies are identified and the pooled effect size estimate thus remains identical after having implemented the trim and fill procedure.

Does anyone know why this could be the case? Could the large sample be an issue?

Thank you!

The funnel plot is copied below for your reference: enter image description here

$\endgroup$
3
  • $\begingroup$ See this Q&A for more information stats.stackexchange.com/questions/236551/… The comments to the question there make several useful suggestions. $\endgroup$
    – mdewey
    Aug 21, 2023 at 13:09
  • 3
    $\begingroup$ What ever else that funnel plot shows it is not small study effects. Most of the asymmetry is in the region of large studies. $\endgroup$
    – mdewey
    Aug 22, 2023 at 14:57
  • 2
    $\begingroup$ I have voted to reopen this as the OP has provided extra detail to clarify what the exact problem is. For the record I do not think this is a programming issue. $\endgroup$
    – mdewey
    Aug 22, 2023 at 15:00

1 Answer 1

3
$\begingroup$

I think the fundamental problem here is that you are replying on evidence from formal statistical tests and methods (Egger and trim and fill) without pausing to look at the plot and ask yourself whether it looks like a text-book plot of small study effects. I do not mean that unkindly, in my experience that is what most people do.

In this case the finding of the Egger test are, probably, being driven by the few small studies to the right hand side of the plot. But the main and most striking feature is the massive asymmetry at the top, where the large studies are. What I would do next is to try to see whether there is any moderator variable which explains what is going on at the top of the plot. This could either be a formal test or subject matter knowledge that they differ in some other way.

$\endgroup$
1
  • 1
    $\begingroup$ There are also issues with dependent estimates - points falling on slightly sloping down lines are typical for Hedges' g values when including multiple estimates from the same sample. $\endgroup$
    – Wolfgang
    Aug 28, 2023 at 11:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.