14
$\begingroup$

The question is as the title says: what is the difference between A/B Testing and Randomized Control Trials?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
13
$\begingroup$

A/B testing seems to be computer geeks terminology, but the idea is of course same. You have an control version of web-page and changed one and you test if difference between some user action rate is statistically significant between versions of pages.

A/B testing tests single feature combination differences when multivariate testing tests different combinations of their interactions.

Improve this answer
$\endgroup$
1
  • 6
    $\begingroup$ To emphasize: there is no actual difference. They're different terms for the same thing. $\endgroup$
    – Kodiologist
    Commented Aug 12, 2016 at 6:08
2
$\begingroup$

As @Analyst (+1) says the basic idea is the same; A/B tests and RCT refer to the same notion.

What is interesting to note is that while in the past the methodological advancements/notions have been primarily a one-way street from RCTs to A/B tests, i.e. from "medical statistics" to "online marketing", lately there have been works in the opposite direction. Obviously, this is partially fuelled by the extensive investing online retailers have on the field; nevertheless a lot of healthcare nowadays has a clear electronic footprint too (wearable sensors being the most obvious example). To that extent, I think that some issues like violations of SUTVA were more aggressively explored by online retailers than healthcare practitioners, as network effects have such a prominent influence in online behaviour. Similarly, some of the most involved and high-value inferential tasks like that of Heterogeneous Treatment Effects (HTE) estimation directly translate to industry goals of personalised marketing and precision medicine respectively. Thus allowing a lot of learnings being transferred from "online marketing" to "medical statistics".

Two prominent papers on this matter are Kohavi et al. (2020) Online randomized controlled experiments at scale: lessons and extensions to medicine and Austrian et al. (2021) Applying A/B Testing to Clinical Decision Support: Rapid Randomized Controlled Trials. Both papers are very insightful, easy to read and help make this functional equivalence between the two terms more clear and well-defined.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.