I'd like to perform a meta-analysis of treatment effects found across >10 studies, but there are varying degrees of sample overlap between these studies (ranging from 0-35% (relative to larger dataset of given pair)). My understanding is that sample overlap would essentially falsely lead to greater precision of the meta-analysed estimate, rather than bias it. I'm however particularly interested in detecting small study bias. So, would a test aiming to detect small study bias (e.g., Egger's test) hold up in the presence of sample overlap?
1
-
3$\begingroup$ Sample overlap is a problem that won't go away by using a specific analysis procedure without individual patient-level data. The next question is how problematic is the overlap, taking into account the incidence of events. For example, let's say the overlap is in people who have failed treatment. That might be a huge issue as some may fail all treatments given to them. The best approach is to use the studies that don't overlap, but if you decide to use overlapping studies make sure you do a sensitivity analysis swapping between which ones go into the analysis and see if the results change. $\endgroup$– abousettaCommented Oct 29, 2023 at 23:17
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Hypothesis testing in case of dependencies will yield inflated Type-I error rates. This also holds for Egger's test that includes the standard error as moderator in a meta-regression model.
Rodgers and Pustejovksy (2021) looked at the statistical properties of the Egger's test and other publication bias/small-study effects tests in the presence of dependencies. They also argue that multilevel meta-analysis and robust variance estimation are alternatives that do maintain accurate Type-I error rates. Hence, one of these approaches can probably also used here.
Rodgers, M. A., & Pustejovsky, J. E. (2021). Evaluating meta-analytic methods to detect selective reporting in the presence of dependent effect sizes. Psychological Methods, 26(2), 141–160. https://doi.org/10.1037/met0000300
Note that the paper is not open access, but a preprint is available here
