How to tell if a covariate modifies the effect if each subgroup has different number of participants? Im working on a meta-analysis of interventional studies that compare between two kinds of interventions.
After pooling the studies together, the heterogeneity using I squared was 0%.
Afterwards, I decided to perform a subgroup analysis to test if a certain covariate modifies the effect. However, I noted that each subgroup contains substantially different number of participants. One contains 100 individuals, while another contains +300.
Is there a reliable way to test for subgroup differences in this situation?
 A: Firstly, very few tests for differential effects in subgroups of randomized trials are affected in their validity by treatment group or subgroup size imbalance (they may of course be affected in other ways, e.g. power). The most natural test would be those based on suitable regression models (e.g. linear regression for continuous outcomes that don't need transformation, logistic regression for binary data, negative binomial regression for count data etc.) adjusting (or stratifying) for study (but assuming numbers are not too small something like a regression model using effect size by subgroup with the standard deviation fixed to be the standard error adjusted/stratified for study would probably also work).
However, whether such an investigation of subgroups is reliable is more of a question. Firstly, remember that you need way more data to look at subgroups (but with a meta-analysis of many studies, I suppose there is at least hope that you could may do something meaningful). Secondly, non-pre-specification, data-driven selection of the subgroup(s) to look into/the direction of the effect of interest and researcher degrees of freedom in how analyses are done (e.g. definition of subgroups, model/test to use etc.) lead to huge potential for false positive findings.
