Are there good alternatives to Cochran's Q test for heterogeneity in meta-analysis?

Question

We used the Q test to determine heterogeneity with p-values <0.10. A highly recognised statistician commented that the Q test has low power and that it should not be used to determine presence of heterogeneity.

My questions:

• Low power is a known issue. Are there clear alternatives? Can the exact version help in scenarios with low sample sizes?

• I cannot easily find alternatives and Cochran's Q is standard in the field (his wording was different and quite strong though). Should the Q test not be used to determine presence of heterogeneity? This seems to contradict many other sources. I have seen the test statistic H^2 but it is derived from Cochran’s Q and it is not commonly used (from what I can see).

I have searched across SO sites and elsewhere (Cochrane's Handbook, Meta-Analysis with R from Guido Schwarzer) but can't find clear answers. Related questions are:

Appropriate homogeneity test for meta-analysis

Subgroups Cochran's Q test

Test for homogeneity in meta-analysis with a large number of studies

https://stackoverflow.com/questions/56362186/exact-cochrans-q-test-in-r?r=SearchResults

Two Study Meta-Analysis: Fixed- vs. Random-effects and heterogeneity

Heterogeneity test in meta-analysis

Answer 1

I would just use a likelihood ratio test between your model with and without random effects (on the papers or specifications) where you retain in both models the usual fixed effects for study type etc.

Some cursory discussion is in the article Interpretation of random effects meta-analyses.

Here is step by step instruction:

1. Estimate a maximum likelihood model, with whatever fixed effects you want to include on study characteristics — usually you will have things like study type (i.e panel vs x section, random trial vs natural experiment), statistical technique (OLS vs logit etc.), sample characteristics (time, sex, age, race, income etc.) and perhaps publication details (time of publication, top-tier journal or not etc.)

2. estimate as per 1., but add random effects on the papers — you are now estimating a mixed model.

3. Apply Wilks' D test to test for significance of the random effects.

4. If (as is almost certain) you find the random effects are significant, you can either use the random effects model, or if you like, use likelihood weighted model averaging over the fixed effects and mixed models.