I am just starting to look at Bayesian statistics and so far I am aware that Bayes factor summarizes some form of evidence of an alternative hypothesis against the null one.

As far as I know we can compare two competing models that use the same data, but is it not possible in any way to apply it to A/B testing?


You can always think about your data as about a single set. In A/B testing it just happens that part of your dataset is related only to the method A and the other part is related only to the method B.

I'd suggest to take a look at: https://nbviewer.jupyter.org/github/CamDavidsonPilon/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/blob/master/Chapter2_MorePyMC/Ch2_MorePyMC_PyMC3.ipynb#Example:-Bayesian-A/B-testing

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.