Subgroups in meta-analysis I am reviewing a dissertation that does a meta-analysis of studies looking at a certain behaviour in rodents.
The author analysed ~20 studies and did a meta-analysis. They then split those studies into sub-categories, depending on the specific methodology that was used in each study (the same thing can be measured in several different ways) and performed one meta-analysis per group.
Finally, they split the original set in a different way (depending on another factor) and performed some further multiple meta-analyses.
I am really not that familiar with meta-analyses, so I am wondering whether this is good practice, or whether they should have included methodology as a factor in their overall analysis instead? If splitting data into multiple subgroups should there be a multiple comparison correction?
 A: For simplicity let us assume there are two groups although the issues are the same for multiple groups. The choices are (a) to run an analysis on each group separately (b) to run an analysis on the whole data-set with a two-level factor as a moderator.
The advantage of option (b) is that you have more data and hence your estimate of $\tau^2$ will be more precise. With 20 studies it was probably not very precise anyway (suggest they give confidence intervals for $\tau^2$ or $I^2$). A slight disadvantage of (b) is that it assumes that $\tau^2$ has the same value in each group but there are options in software to relax that assumption. There is detail on how to do this in R using the metafor package here.
If the split was based on theory then there is no real multiplicity issue here but if it is data-driven then caution is obviously needed.
A: There definitely should be a multiple comparison correction.
Meta-analyses are...generally not of the highest quality, with regards to statistical soundness.
A: It depends whether the data obtained using different methodologies can be statistically compared. Normally in a MA you first show the results overall and then you run a sensitivity analysis e.g. stratified analysis.
