1
$\begingroup$

I conducted a network meta-analysis in frequentist framework using the R package netmeta (https://cran.r-project.org/web/packages/netmeta/netmeta.pdf), statistical details of this package are available here (https://www.ncbi.nlm.nih.gov/pubmed/26053424). This package generates both fixed effect and random effects results by default. Surprisingly, for my analysis, both outputs are exactly identical. How can I explain this? The number of studies in my analysis is 10. Initially, I included 12 studies in the analysis, which resulted in a high statistical heterogeneity (I^2 >95%). I removed two of the studies with high heterogeneity reducing the number to 10 and a very low heterogeneity (tau =0; I^2=0). Could this very low heterogeneity be the reason for exactly identical outputs? Is this approach of excluding studies to manage heterogeneity valid? As I understand, an exhaustive NMA should include all available evidence.

Studies are clinically homogenous and all are randomised trials. Most of the treatments are being compared against Placebo.

Improve this question
$\endgroup$
11
  • $\begingroup$ Low heterogeneity? $\endgroup$
    – mdewey
    Commented Jul 13, 2019 at 12:30
  • $\begingroup$ ndeed, heterogeneity is very low (tau^2 = 0; I^2 = 0%). Actually, I removed two of the studies from this analysis that were resulting in very high heterogeneity (I^2 >95%). I did pairwise meta-analyses to identify those studies and removed them from the main NMA, kept them for a sensitivity analysis, which I haven't done yet. Could you please suggest if this approach is reasonable? Could you please also elaborate more on consequences of having low heterogeneity? $\endgroup$
    – Prabhakar
    Commented Jul 13, 2019 at 18:03
  • 1
    $\begingroup$ If heterogeneity is very low and the number of studies limited I think it is perfectly OK. Indeed, report both results and highlight this finding. It could also depend on the fact that your evidence network is mostly star-shaped (amounting mostly to an adjusted indirect comparison...). $\endgroup$
    – Giuseppe Biondi-Zoccai
    Commented Jul 14, 2019 at 11:29
  • 1
    $\begingroup$ Without a summary of results and a brief picture of your study, it is difficult to answer your concerns. $\endgroup$
    – user10619
    Commented Jul 14, 2019 at 14:25
  • 1
    $\begingroup$ I don't recommend to exclude studies based on their findings, but only given their design features (eg randomized trials included, non-randomized trials excluded). A star-shaped network is a network where most treatments have been compared to a single agent (eg placebo). $\endgroup$
    – Giuseppe Biondi-Zoccai
    Commented Jul 15, 2019 at 13:10

1 Answer 1

Reset to default
1
$\begingroup$

Firstly, it is pretty dubious to exclude studies, because their results mismatch your model. If the studies fit all your inclusion criteria, I suspect many people will feel uncomfortable about removing them to reduce betwen study heterogeneity in outcomes.

Secondly, if the between study variability is estimated to be zero (on the boundary of the parameter space) in a random effects model, there are multiple ways of dealing with that. One common option in maximum likelihood approaches is to treat the parameter as if it were known to be exactly zero, which is what the software you use seems to do.

With Bayesian approaches (either maximum-a-posteriori with boundary avoiding priors or with proper priors and looking at other posterior summaries) this is - in my personal opinion - handled in a more natural manner, but then you of course have to set priors and we know that in a meta-analysis with few studies the prior for the between-study heterogeneity can be quite influential.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.