2
$\begingroup$

If the results from a network meta-analysis using standard means differences (Hedges g) and using both direct and indirect evidence finds that:

  • Treatment A is not significantly less/more effective that Treatment B (A = B)
  • Treatment A is not significantly less/more effective than Treatment C (A = C)
  • Treatment B is significantly more effective than Treatment C (B > C)

In short:

  • A = B
  • A = C
  • B > C

Then would it be fair to at least consider that this might indicate the assumption of transitivity has been violated?

I found this post that I think is similar to my query and it mentions the Condorcet Paradox and the violation of transitivity.

EDIT:

To add the actual SMDs:

  • AB Comparison (g = -0.27; CI = -0.81 to 0.28): Favours B
  • AC Comparison (g = -0.29; CI = -0.29 to 0.14): Favours A
  • BC Comparison (g = 0.56; CI = 0.11 to 1.0): Favours B
Improve this question
$\endgroup$
6
  • 2
    $\begingroup$ "Significantly more" generally is not a valid partial order relation, that's all. You're not allowed to assume transitivity: it either is true or not; and it's not. In particular, "=" is not an equivalence relation (if, by "=", you mean "not significantly different"). $\endgroup$
    – whuber ♦
    Jul 1, 2020 at 19:17
  • 4
    $\begingroup$ Remember that an insignificant p-value is not acceptable of the null hypothesis!You're in a situation where $A$ can't be distinguished from $B$ and $A$ can't be distinguished from $C$. Notice my phrasing: "can't be distinguished" and not "is equal".So put them in order from highest to lowest: $B$, $A$, $C$. You could have that you can't tell the middle value apart from either extreme, but you also get that the extremes are different. Phrased this way, it does not sound so weird. $\endgroup$
    – Dave
    Jul 1, 2020 at 19:25
  • $\begingroup$ Thanks for the replies. My use of language around statistics definitely needs a brush up. Dave, I can see the point you're making and that does help me mentally make more sense of the findings. In terms of making any kind of statement that the average person (me included) would understand about this kind of finding, the authors of this study made treatment recommendations that have been adopted (in reality A is Group Therapy, B is Individual Therapy, and C is Placebo) and they recommended that Individual Therapy (B) should be offered and that Group Therapy (A) should not - does this seem fair? $\endgroup$
    – Strooby
    Jul 1, 2020 at 19:46
  • 2
    $\begingroup$ What they're saying is that they can't tell if individual therapy is more (or less) effective than group therapy and that they can't tell if group therapy is more (or less) effective than the placebo, but they observe that individual therapy is more effective than placebo. $\endgroup$
    – Dave
    Jul 1, 2020 at 19:51
  • 1
    $\begingroup$ That perhaps warrants a separate question. $\endgroup$
    – Dave
    Jul 1, 2020 at 21:15

1 Answer 1

Reset to default
5
$\begingroup$

One way to quantify the departure from transitivity is to add up the A-B, B-C, and C-A differences.

Under transitivity, this will have mean zero, and variance equal to the sum of the three variances, so you can see how big the departure is both in clinical terms and compared to the uncertainty.

That was the original approach in the paper that coined the name 'network meta-analysis', only using a linear mixed model to do it simultaneously for all the loops in the network.

Improve this answer
$\endgroup$
2
  • $\begingroup$ That's really interesting. When adding the SMDs up for each effect (-0.27; 0.56; -0.29) they come to zero. They used a Bayesian random-effects network meta-analysis. $\endgroup$
    – Strooby
    Jul 2, 2020 at 9:11
  • 1
    $\begingroup$ If they come to exactly zero, the estimation procedure may have enforced transitivity. $\endgroup$
    – Thomas Lumley
    Jul 2, 2020 at 23:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.