I am conducting a meta-analysis and I'm trying to ascertain how to test for publication bias and heterogeneity.

However, I have used multiple datasets to calculate effect sizes, and therefore I'm unsure how to go around this.

For example, in comparing locomotory behaviour in animals of the taxonomic Order, Artiodactyla in the zoo and the wild, I have taken the mean of locomotory behaviour in 11 separate studies in the wild and 7 separate studies in the zoo to calculate the effect size (Hedges' g).

As I understand, publication bias would require me to calculate the effect size from each individual study but this isn't possible when each separate study looks at behaviour either in the wild or the zoo and not both.

As I also understand, using cochran's Q test for heterogeneity would require comparing samples of the same size, which of course for this data is not possible, since the behaviour performance is taken from different studies.

Therefore, I'm not even sure if it is possible to test for publication bias or heterogeneity. Any help with this would be appreciated.


2 Answers 2


I don't see any way that you can calculate heterogeneity or bias; I'm not sure this even really qualifies as "meta-analysis". You don't have any effect size measures to combine. And I'm not sure whether it is a good idea to average across different studies. And, as I understand it, "publication bias" is when studies that are statistically significant are more likely to be published than those that are not. But it seems like the studies you are combining are purely descriptive and, therefore, this doesn't arise.

If you have (or can get) the raw data, why not just do a regression where locomotor behavior is the dependent variable and location (wild vs. zoo) is the independent variable?

Failing that, you might just call it a quantitative review and look, informally, at each study and its results, and calculate descriptive statistics and make some graphs.


If you want to do a meta-analysis here you need to compute an effect size from ecah study. It is not clear to me exactly what that would be from your description but I assume it would be the mean with its standard error. You can then combine the 18 measure in a meta-regression with location *zoo versus wild) as a moderator. The coefficient for the moderator will give you the estimated mean difference between the two locations. The usual tests for heterogeneity will work.

The issue of small-study effects (what you called publication bias) is more interesting and I have already covered it in the context of the meta-analysis of proportions in my answer in this Q&A

  • $\begingroup$ I don't think these means are effect sizes, certainly not for what the OP wants to study. In fact, they don't have any effect sizes, so, they can't do a meta-analysis. $\endgroup$
    – Peter Flom
    Jan 14 at 18:30
  • $\begingroup$ @PeterFlom it is perfectly possible to meta-analyse non-comparative studies like the ones the OP has. You can analyse proportions (usually transformed), means and less commonly variances, incidence rates and probably some others I have not come across. $\endgroup$
    – mdewey
    Jan 15 at 13:50
  • $\begingroup$ OK, but not for what the OP wants to measure; at least, I don't see how. That is, if your goal is to estimate a proportion, you can meta-analyze proportions. But OP wants to estimate a difference. $\endgroup$
    – Peter Flom
    Jan 15 at 14:44
  • 1
    $\begingroup$ @PeterFlom which is why I suggested meta-regression with a moderator for location of the animals. It would be a non-randomised comparison of course but given the nature of the moderator it is hard to see animals being randomised to wild and zoo. $\endgroup$
    – mdewey
    Jan 15 at 15:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.