4
$\begingroup$

I am new to meta analysis and how I understood the terminology is that there are actually two ways of performing a meta analysis. Let's consider 5 clinical studies with fixed effects. Fixed effects in terms of the same medical treatment as well as demographic details of the participants. One way of analysing these data would be to pool all 5 studies together to obtain a very large study to increase the power to detect the effect of the medical treatment. The other would be to try to detect the effect in each analysis separately and then determine the average effect across the studies. As I understood meta analysis, both seem to be reasonable techniques. However, can anyone tell me pro's and con's for both techniques? When should I use which method? I would assume the results to be pretty similar anyhow or is that wrong to assume?

$\endgroup$
  • 2
    $\begingroup$ Hi, I'll give a more detailed answer later, but simply put they are not the same. You should first be measuring the treatment effect within a study and then pooling across studies. The other way is known as the 'naive method' and is dangerous as it can give invalid results. Have a look at the wikipedia page for Sampson's paradox for some examples (en.wikipedia.org/wiki/Simpson%27s_paradox). $\endgroup$ – abousetta Oct 14 '13 at 2:47
  • $\begingroup$ what is the effect-size - d-statistic or any other? $\endgroup$ – Subhash C. Davar May 3 '16 at 12:31
0
$\begingroup$

Converting @abousetta's comment into an answer since s/he obviously did not have time to come back.

Simply put they are not the same. You should first be measuring the treatment effect within a study and then pooling across studies. The other way is known as the 'naive method' and is dangerous as it can give invalid results. Have a look at the wikipedia page for Simpson's paradox for some examples (en.wikipedia.org/wiki/Simpson%27s_paradox).

$\endgroup$
0
$\begingroup$

There is often very little difference between

  1. pooling all the data on a patient-level, running a statistical model that adjusts for study (e.g. if we are talking about a GLM by including a trial fixed effect or a study stratum in the model), and
  2. analyzing each study separately and combining the estimates using a fixed effects meta-analysis method (e.g. inverse variance).

Similarly, there is not much difference between having a random treatment effect for each study in (1) and combining estimates in (2) using random effects meta-analysis methods. Option (1) is also called "individual patient data"-meta-analysis and option (2) is what is often meant when people say "meta-analysis" without any qualifiers.

Differences mostly occur either due to small sample/sparse data effects or due to how covariates are adjusted for (effect assume to be the same in the case of (1) and allowed to be different in the case of (2)).

Where you can get substantial differences (and very questionable conclusions) is when you do not adjust for study in option (1). That is generally very much not advisable, but can occasionally under special circumstances be okay (e.g. all trials have randomized patients to the same treatment groups with identical allocation ratios, in which case it may just be inefficient).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.