1
$\begingroup$

I have a question about the use of publication bias modeling approaches in meta-analyses of proportions.

The traditional approaches of assessing publication bias, such as the rank correlation test, Egger’s regression model, and weight function approaches have all assumed that the likelihood of a study getting published depends on its sample size and statistical significance (Coburn and Vevea, 2015). Although it has been confirmed by empirical research that statistical significance plays a dominant role in publication (Preston et al., 2004), this is not entirely the case. Cooper et al. (1997) have demonstrated that the decision as to whether to publish a study is influenced by a variety of criteria created by journal editors regardless of methodological quality and significance, including but not limited to, the source of funding for research, social preferences at the time when research is conducted, etc. Obviously,the traditional methods fail to capture the full complexity of the selection process.

In practice, authors of meta-analyses of proportions have employed these methods in an attempt to detect publication bias. But, studies included in meta-analyses of proportions are non-comparative, thus there are no “negative” or “undesirable” results or study characteristics like significant levels that may have biased publications (Maulik et al., 2011).

Therefore, in my opinion, these traditional methods may not be able to fully explain the asymmetric distribution of effect sizes on funnel plots. It is also possible that they may fail to identify publication bias in meta-analyses of proportions in that publication bias in non-comparative studies may arise for reasons other than significance.

I'm not sure if my reasoning is correct. Can someone shed some light on this? If someone could point me to some papers regarding this topic, that'd be wonderful.

References: Coburn, K. M., & Vevea, J. L. (2015). Publication bias as a function of study characteristics. Psychological methods, 20(3), 310.

Cooper, H., DeNeve, K., & Charlton, K. (1997). Finding the missing science: The fate of studies submitted for review by a human subjects committee. Psychological Methods, 2(4), 447.

Preston, C., Ashby, D., & Smyth, R. (2004). Adjusting for publication bias: modelling the selection process. Journal of Evaluation in Clinical Practice, 10(2), 313-322.

Maulik, P. K., Mascarenhas, M. N., Mathers, C. D., Dua, T., & Saxena, S. (2011). Prevalence of intellectual disability: a meta-analysis of population-based studies. Research in developmental disabilities, 32(2), 419-436.

$\endgroup$
1
$\begingroup$

As you suggest in your question people usually examine small-study effects because they feel that small studies with a result in one direction will be less likely to come to hand. In the case of non-comparative studies it is less clear how this is going to arise. The more likely situation in non-comparative studies is that there will eventually arise in the community a consensus about what the true underlying value might be and so small studies with results in either direction will not come to hand. The funnel plot in such cases would come to resemble a narrow strip and the regression tests would fail to detect anything of interest. One example which comes to mind it community prevalence studies of dementia. Early studies produced variable values but as techniques improved with the adoption of standardised measurement methods and international nosologies the value found converged on approximately 5% in samples 65+ years. A colleagues whose study ahd found such a value in a country where there was an absence of previous results was quite pleased but as I remarked to him at the time if he had found a very different value he would have found it difficult to get it published and it would have been treated with scepticism.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.