# Modern statistics (with a focus on hypothesis testing) textbooks balanced in terms of rigor and the applied aspect

I work as a data scientist at a product-oriented company. I guess I will not surprise anyone on this website that it's getting more and more popular for companies with enough data and resources to perform lots of experiments and then make even the smallest decisions based on the results.

Recently, I have been trying to delve into hypothesis testing and A/B tests, in particular, so that I can understand the underlying mathematics, not just use one of the popular calculators in order to calculate the necessary sample size. I have noticed that it's quite a common (and pragmatic) way to get the job done, but I would like to understand every single formula and calculation. I have a Master's degree in applied math and computer science so, as you might understand, just being told a formula doesn't feel right.

One of the most recommended books on A/B testing is this one: https://www.goodreads.com/book/show/51635906-trustworthy-online-controlled-experiments?from_search=true&from_srp=true&qid=IsvcUgzDqk&rank=1

The author is a legend and the book has lots of useful information in terms of methodology, but the mathematical aspect is rather dumbed down. For example, when discussing the sample size, there is a rule of thumb without any derivations that you can use in order to get an estimate of the required sample size.

I have found this website useful, for example: https://online.stat.psu.edu/stat500/

It has several courses with hypothesis testing included, but it's not comprehensive and a lot of derivations are missing as well.

There is the classical textbook on hypothesis testing by Lehman and Romano: https://books.google.co.uz/books?id=Y7vSVW3ebSwC&source=gbs_similarbooks&redir_esc=y

Funnily enough, I find it to be the opposite. It's very rigorous and yet I cannot get rid of the feeling that the authors don't care about the practical side of the theory.

I am wondering if there are any great and modern resources (preferably, books) that strike balance between rigor and being practical. I have heard that it might make sense to look for biostatistics books or trials in medicine in general because hypothesis testing is super relevant in that field, but anyway I don't know any good resources except for those with good reviews on Amazon.

• I think your problem is that you're comparing introductory level materialto graduate level mathematics texts. The former are limited to algebra, and one or two lines of code. The latter require calculus, even real analysis, and advanced programming skill. What is "practical" to you depends what level you're at. When you're an expert, you need methods to assess the need for, employ, and interpret Bayesian, resampling, non-parametric, etc. statistics. Commented Apr 19, 2021 at 20:34
• I have nothing against calculus and even real analysis. I took courses in measure theory and functional analysis, but I am definitely not an expert in statistics. Hence the question ;) Commented Apr 19, 2021 at 20:36
• The first book you mention has two other authors, Diane Tang and Ya Xu, who are also prominent practitioners and co-authors on many of the papers that the book covers. Commented Apr 21, 2021 at 0:26