1
$\begingroup$

In meta-analyses, funnel-plot asymmetry tests are usually based on the association between effect size and its standard error (or variance, that in case of rank-tests are the same) or, alternatively, sample size (that is strongly associated with SE but prevents the problem of the association between the estimates of effect and of its SE). Basically, they test whether sample size is associated with the estimated effect: what we check is whether smaller sample sizes have a different (we suspect higher) estimated effect in average. However, the mechanism underlying publication bias is assumed to be the one of asymmetry: in fact, we talk about "funnel-plot asymmetry" tests. Thus, why aren't general asymmetry tests (based on the first and third moment, more rarely on other centraly tendency measures) used? At the end of the day, what we suspect is that we are missing studies that are on one tail (i.e., among the ones with lower sample size, or with higher variance, only those above/below the median value), and such one-sided lack of studies would lead to a difference between average and median and to a third central moment away from zero in the distribution of published studies, so why aren't such (in)equalities tested?

EDIT: To reply to Medwey's answer, I added a comment to clarify that I talked about the third moment and the difference between mean and median just thinking at the tests I know, but my question actually is about the lack of tests specifically focusing on tails.

$\endgroup$
0
$\begingroup$

One of the assumptions of the standard meta-analysis model is that the effect sizes have a Gaussian distribution. It would be possible to examine the distribution of the effect sizes (or the residuals from the model if a meta-regression is at issue) and perhaps this should be done more often. However it would be examining a different feature of the data-set than the regression tests for small study effects.

$\endgroup$
  • $\begingroup$ Thank you, Mdewey. The assumption is the Gaussian distribution, that we would observe in case no study were excluded due to publication bias. Since the alternative hypothesis is that we are basically missing studies in one of the two tails of the distribution, I wonder why no tests with an alternative hypothesis based on the comparisons of the two tails are used. If I well understand, all available tests check whether the estimated effect is associated with its standard error/sample size, regardless of whether this is due to the behaviour at extreme values or at other issues. $\endgroup$ – Federico Tedeschi Mar 7 '18 at 14:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.