I am just starting to look at Bayesian statistics and so far I am aware that Bayes factor summarizes some form of evidence of an alternative hypothesis against the null one.

As far as I know we can compare two competing models that use the same data, but is it not possible in any way to apply it to A/B testing?

  • 1
    $\begingroup$ It might help to come up with some example of what kind of A/B test you have in mind so people can explain how to use Bayes factors for a specific problem. $\endgroup$
    – Bernhard
    Commented Mar 22, 2021 at 12:10

1 Answer 1


You can always think about your data as about a single set. In A/B testing it just happens that part of your dataset is related only to the method A and the other part is related only to the method B.

I'd suggest to take a look at: https://nbviewer.jupyter.org/github/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/blob/master/Chapter2_MorePyMC/Ch2_MorePyMC_PyMC3.ipynb#Example:-Bayesian-A/B-testing


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.