0
$\begingroup$

Since a fixed-effects meta-analysis assumes that the studies are all estimating the same underlying effect, when the Cochrane RevMan software reports a Q and $I^2$ statistic with a fixed-effects analysis what exactly are they reporting? The manual says:

Note that a random-effects model does not ‘take account’ of the heterogeneity, in the sense that it is no longer an issue (I am not sure if this is relevant!).

Are these numbers just what you would get with the data if you were to do a random-effects analysis? But then would not the weighting of the studies be different from that in the fixed-effects analysis making it potentially misleading with the fixed-effects forest plot?

$\endgroup$
1
$\begingroup$

You have two different questions here: 1) What do Q and I2 represent, and 2) what is the difference in weighting between a fixed-effect model and a random-effects model.

Q1) In their simplest form, these statistics represent the amount of heterogeneity between the different study data in the meta-analysis. Higher values represent more heterogeneity, which may be due to random sampling error or represent actual differences in the studies. Higgins BMJ paper is a good starting point to explain both (BMJ. 2003 6;327(7414):557-60).

Q2) Random-effects model incorporates the Tau (a measure of heterogeneity), which will change the weight that each study gets and depending on the amount of heterogeneity may widen the confidence intervals. Having said that, it's not an automatic 'correction' of heterogeneity and that's what the quote you mentioned was referring to. In the past, people would report that they would use a fixed-effect model if there was minimal heterogeneity and a random-effects model if there was more heterogeneity. The decision to use a fixed or random model should be made a priori depending on the similarity of the studies, outcome measures, etc., and not depending on the measure of heterogeneity.

$\endgroup$
2
  • $\begingroup$ Thanks for this; but I am still not clear why RevMan gives heterogeneity statistics for a fixed-effects model, which as I understand it does not really allow for heterogeneity to exist? Thanks T $\endgroup$ – Ted Jul 10 '15 at 14:05
  • $\begingroup$ Fixed-effect model does not incorporate heterogeneity into the model, not that it doesn't allow it. You may want to read up on fixed-effect vs. random-effects models. There's a section in the Cochrane Handbook and numerous papers on the topic. $\endgroup$ – abousetta Jul 13 '15 at 15:16
0
$\begingroup$

My understanding is that random effects models try to take into account clinical and methodological heterogeneity between the study populations and interventions, and is more conservative, while fixed effects models do not.

I think that the $I^2$ value is a measure of statistical heterogeneity which is not entirely synonymous with clinical or methodological heterogeneity. E.g., heterogeneity can be seen on the forest plot when the confidence intervals do not all overlap but the studies might have similar clinical and demographic characteristics or vice versa.

See this paper.

Also, see the Cochrane handbook.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.