I am conducting a systematic review to compare the effect of intervention A vs. B on outcome Y (A, B, Y binary variables). So far the collected studies are categorized as below:

  1. 7 studies on both intervention A's effect on Y (A x Y) and B x Y
  2. 60 studies on A x Y
  3. 9 studies on B x Y

My questions are as follows:

  1. In performing meta-analysis to compare A x Y and B x Y, is there any benefit (in terms of statistical strength of conclusion) to only pool the 7 studies that simultaneously study the effect of A & B on Y, in comparison to pooling 60 studies on A x Y and 9 studies on B x Y?

  2. Of the 7 studies that simultaneously look at A x Y and B x Y, 4 studies found the effect of A x Y insignificant, but all found significant effect of B x Y. The effect sizes are similar. What are the risks of concluding that B has more significant effect on Y than A? I am aware of the risks associated with vote counting method, but in this case I am dealing with findings from identical set of studies.

I would very much appreciate any input or pointer to relevant literature/keywords. Thank you in advance.

  • $\begingroup$ Hi, Just for clarification (from my understanding), you have two interventions (A and B) and you want to see their comparative effect on outcome (Y). In this case, statistics aside, there is a monumental difference between studies comparing A vs. B (in the same study) versus studies only comparing A and studies only comparing B. The reasons are simply there you would have to make assumptions on whether the patient populations, baseline characteristics, etc. are the same in studies only comparing the effect of intervention A or only intervention B. (to be continued)... $\endgroup$ – abousetta Nov 17 '13 at 16:18
  • $\begingroup$ Therefore from a pure theoretical stance, you would want to pool studies comparing A vs. B by themselves; studies comparing intervention A only by themselves; interventions comparing intervention B only by themselves. $\endgroup$ – abousetta Nov 17 '13 at 16:21
  • $\begingroup$ If provide more information on your interventions and outcome of interest, I can elaborate further. Hope this helps. $\endgroup$ – abousetta Nov 17 '13 at 16:21
  • $\begingroup$ Thank you so much for your comment, @abousetta. My Y is an incidence of illness (yes or no) and A is a home environment characteristic parameter (yes or no) and B is a neighbourhood environment characteristic parameter (yes or no). I am hoping to determine which one of A and B is more significant on Y. $\endgroup$ – Tiffany Nov 17 '13 at 21:54
  • $\begingroup$ If I only use 7 studies that simultaneously examine the effect of A and B on Y in a multivariate analysis, I would be pooling the effect size (odds ratio) of A on Y separately from B on Y. Is there any additional risk of bias if I only pool these 7 studies and not include the other studies (60 studies on A x Y, 9 studies on B x Y)? The other studies are similar to the 7 studies in terms of study design.. except that they don't control for A & Y simultaneously. $\endgroup$ – Tiffany Nov 17 '13 at 21:54

So that this old question does not go unanswered and based on the further details revealed in the comments this is a question which could possibly be answered by a network meta-analysis. There are a number of helpful questions and answers tagged . As @abousetta remarked in a comment the validity of this is going to depend strongly on whether studies which only used A or only B are comparable with those which used both.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.