Reporting results of fixed effect MA alongside results of random effects MA

Question

I have conducted meta-analyses for a Cochrane systematic review. I was comfortable pre-specifying and justifying whether a fixed effect or random effects approach will be used in the MA. Having completed the meta-analyses and used random effects models in some instances my co-author is requesting that I include results from fixed effect models alongside the random effects results.

Given that I pre-stated the analysis approach (although this is an update so there is no registered protocol per-se), then reporting both RE and FE results seems to undermine the purpose of pre-stating the analysis approach.

Should I simply include the FE results, and be clear that they come under sensitivity analysis or stick to my guns and only report what I said would be done? My co-author's view is that FE is more reliable when there is no heterogeneity or there is suspected publication bias.

Answer 1

The debate over FE vs. RE often reflects around the lack of understanding of the concepts behind the models. To make matters worse, the Cochrane handbook only briefly touches on this (9.4.2 Principles of meta-analysis) and provides some guidance on whether or not the posthoc decision to use the FE or the RE was the correct one (10.4.4.1 Comparing fixed and random-effects estimates). From the Handbook: "The combination of intervention effect estimates across studies may optionally incorporate an assumption that the studies are not all estimating the same intervention effect, but estimate intervention effects that follow a distribution across studies. This is the basis of a random-effects meta-analysis (see Section 9.5.4). Alternatively, if it is assumed that each study is estimating exactly the same quantity a fixed-effect meta-analysis is performed."

In only the rarest types of meta-analyses (e.g. same trial repeated multiple times with the same exact protocol) could we assume that the studies are all estimating the same intervention effect, with between-study differences in results attributed to random error. On the other hand, we typically combine studies that use similar populations, similar interventions, similar comparisons, similar doses, etc. assuming that the studies have a reasonable amount of clinical homogeneity allowing their data to be pooled together.

Therefore, IMHO, the decision to use the FE or RE model should be performed at the protocol stage since the actual results do not play a factor in the model choice. Having said that, as noted in section 10.4.4.1 of the Handbook, this choice may need to be reversed after seeing the results of the meta-analysis; if there is suspicion of small-study effect skewing the pooled results.

So, in closing, unless there is justification for the use of FE model, the RE model should be the default choice when conducting a meta-analysis from different study results. Of course, that doesn't mean you can't test test the robustness of the analyses in a sensitivity analysis (FE model). Actually a lot of Cochrane groups request this.

Answer 2

To answer this question, one must first consider when the meta analytic estimates from a RE model differ from a FE model. If there is zero between-study heterogeneity, but one uses the RE model, the estimate and interval will be equivalent to a FE model. This is because the FE is assuming 0 heterogeneity. Now, when the estimates differ, this is because there is some degree of variability in the individual effects.

With this in mind, you can ALWAYS assume a RE model, and if there happens to be zero heterogeneity, then your estimates will accurately reflect that fact. The opposite is not true, however, as fixing tau to zero with a FE model "forces" upon your data what is likely an untenable assumption.

With this in mind, if your estimates differ by some amount, this is because there IS between-study variability.

In regards to another answer posted here, I do not see a FE model as a sensitivity analysis. If anything, you will be more likely to reach significance (narrower interval due to possibly underestimating tau). The RE could be considered a sensitivity analysis, but should really just be the default for the above stated reasons.

Answer 3

The distinction here has been ably set out by @abousetta but it is perhaps wrth stressing that neither approach can be regarded as completely satisfactory. The RE estimate is very affected by the smaller studies which are often considered less reliable whereas the FE estimate of the confidence interval is too narrow. The proble has been addressed by Henmi and Copas in an article here entitled "Confidence intervals for random effects meta--analysis and robustness to publication bias". They use the central estimate from the FE model but with a confidence interval which accounts for $\tau^2>0$. The method is available in the metafor package in R. Disclaimer 1: I corrected a few typos in the code and made it available for the package author. Disclaimer 2: I am not claiming that this method is going to solve all the problems of meta-analysis but it is an intriguing option.