4
$\begingroup$

I'm trying to learn about A/B testing with more statistical depth. I have a good idea of the basics, however, I am struggling to find decent resources that discuss all the mathematical technicalities and properly prove formulas and methodology. I've been jumping between youtube videos and suggested industry books, but none have provided the depth I'm looking for and some give different or contradicting information on the details.

I'm wondering if anyone could suggest some reliable free resources that discuss A/B testing in full statistical depth, or if anyone is willing to do a full breakdown of the topic. If there are any suggestions on how I can better learn the material, I'd appreciate it.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
3
$\begingroup$

A few ideas:

  1. Browse the Q&As here on our site with the tag (and maybe ).
  2. Consult Experimental Design textbooks like Montgomery
  3. Consult the clinical trials literature. The book by Pocock is very good.
  4. See some webpages such as https://vwo.com/ab-testing/ and https://cxl.com/blog/ab-testing-guide/
Improve this answer
$\endgroup$
1
$\begingroup$

A few suggestions;

Note that A/B is nothing truly unique (it is just web developpers rediscovering statistics); so any good intro statistical book would do the job; for this, you may want to look at this post on CV or this one.

Improve this answer
$\endgroup$
3
  • $\begingroup$ As I read more about it, it just seems like basic 2-sample hypothesis testing where the appropriate test and statistic must be selected. Is this correct? $\endgroup$
    – jen
    Commented Oct 15 at 17:52
  • $\begingroup$ Basically yes (sometimes it goes beyond 2 samples (A/B/n) but it is still hypothesis testing (Anova etc...)). Hence my suggestion to also look at introductory general statistics textbooks. $\endgroup$
    – jginestet
    Commented Oct 15 at 18:42
  • $\begingroup$ makes sense, i was looking at my old intro to stats book and its helped a lot $\endgroup$
    – jen
    Commented Oct 15 at 23:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.