Test for homogeneity in meta-analysis with a large number of studies

Question

So I am conducting some literature searches on behalf of someone. I stumbled across a meta-analysis which used 23 studies. It tested the homogeneity of the studies used, using what I think is a Cochran's Q test. The output is as follows:

Heterogeneity: $\hat{\tau}^2 = 0.11$; $Q = 35.59$, $df = 13$ ($P = 0.0007$); $I^2=63%$

The issue is that I think this output means that these studies fail the test of homogeneity and thus a meta-analysis is not appropriate. Am I incorrect in this?

My second question is: Does increasing the number of studies used increase the heterogeneity anyway and thus might it be the case that this test was going to be significant anyway due to sample size?

Any help in understanding this would be greatly appreciated.

Answer 1

The null hypothesis that is tested by the Q-test is that the true effects/outcomes are homogeneous. Since the test is significant, we would reject that null hypothesis and conclude that the true effects are heterogeneous.

I would not say that the studies "fail" the test, but phrasing is a matter of taste. More importantly, there is no reason to consider a meta-analysis inappropriate just because the test is significant. I don't know what kind of analyses the authors did, but the common strategy is to account for heterogeneity by fitting a random-effects model, which provides us with an estimate of the average true effect (and an estimate of the amount of variance in the true effects -- that is in fact the $\hat{\tau}^2$ value).

With respect to the second question: Increasing the number of studies does not automatically increase the amount of heterogeneity. However, the power of the Q-test to reject the null hypothesis increases as the number of studies increases. But this assumes that the true effects are really heterogeneous. If they are homogeneous, then the chances that you will reject the null hypothesis is not influenced by the number of studies.