I did a science PhD doing applied bayesian statistics, and I liked it so much that I decided to shift to statistics. For the last 2-3 years have been working at a firm that has more of a frequentist focus. It seems to me that the bayesian approach (at least what I learned) has an underlying theory that was laid out and I could understand.

Now I find this - more classical - field fascinating but I am looking for some resources to get a unified view of the matter. For example, I am learning and applying anovas, F-tests and chi-squares, thinking about degrees of freedom, etc but part of me is looking for deeper understanding that would make it more than just methods. Does anyone have any suggestions on books or more resources that provide such a unified, deeper focus?

  • $\begingroup$ If you're looking for underlying theory I think you'll probably be disappointed. The "frequentist" approach is more based on intuition and heuristics than mathematical rigor. $\endgroup$ – dsaxton Aug 11 '15 at 15:43
  • $\begingroup$ Have a look at my answers here: math.stackexchange.com/questions/735075/… math.stackexchange.com/questions/468359/… and tell us if you found (some) of those useful. For you I think especially the one by David Cox. $\endgroup$ – kjetil b halvorsen Aug 11 '15 at 16:24
  • 4
    $\begingroup$ @dsaxton What support do you have for that comment? It sounds remarkably inconsistent with the works of Wald, Kiefer, et al. who have created a rigorous foundation for the "frequentist" methods using decision theory. That suggests you might be attacking some kind of straw man definition of "frequentist," which seems to be rather gratuitous (as well as beside the point). $\endgroup$ – whuber Aug 11 '15 at 16:45
  • $\begingroup$ Ramiro, some good suggestions have appeared at stats.stackexchange.com/questions/363, stats.stackexchange.com/questions/66051, and stats.stackexchange.com/questions/17899, inter alia. $\endgroup$ – whuber Aug 11 '15 at 16:47
  • $\begingroup$ You could also check out Berger and Casella's Statistical Inference. $\endgroup$ – Placidia Aug 11 '15 at 17:35

The Introduction to Statistical Inference by Kiefer, mentioned in a comment by ramiro, is a great candidate for this and my own question. But since ramiro's comment is folded and my question is marked as duplicated, I decided to add an answer here so that more people would know this great book.

The book perfectly fulfills the first three requirements listed in my question. Specifically:

  • After a brief description of what is statistical inference in plain English (Chapter 1), it introduces the decision theoretic framework of statistical inference (Chapter 2&3), along with many insightful remarks. Chapter 4 is amazing. It presents most of the principles of statistics: Bayes, Minimax, Admissibility, Unbiasedness, Maximum Likelihood, Method of Moments, along with criticisms and connections.
  • Afterwards, it goes into Linear Regression (Chapter 5), Point Estimation (Chapter 7), Hypothesis Testing (Chapter 8), and Confidence Intervals (Chapter 9). Sufficiency (Chapter 6) and Asymptotics (Section 7.6) are also covered. All these are packed concisely within about 300 pages.
  • The prerequisites are only Calculus and Linear Algebra, the latter is optional if you omit '5.4 Analysis of the General Linear Model'. Also, it avoids formalization whenever possible. That said, the mathematical maturity required to follow the (often informal) reasoning is actually more than most of introductions to statistics I've tried (e.g. Casella & Berger, which is usually considered as advanced undergraduate or graduate level). You need to devote many thoughts into nuances of statistical ideas that are rarely found in books of this level.

An additional advantage is the good exercises. They are not simple repetitions of the examples, but rather like exam problems. They are broken into several sub-problems to guide the readers, and there are also hints. A unique feature is the the author's suggestions of which problems you should do as a minimum.

Some downsides:

  • Although the exercises are well-designed and you can solve most of them independently, it would be better if there are solutions.
  • Since the book was written in the 70's, the notation is a bit weird and it takes some time to get used to.

Finally, maybe this should not be count as a problem of the book itself, but as mentioned in my another question, I think it is a big plus if one can get to know some computational techniques and tools from the very beginning of learning Statistics.

  • $\begingroup$ I would be very happy to know other books (or supplements) that address the mentioned problems, at the same time preserve the advantages of Kiefer's. $\endgroup$ – wpzdm May 10 '19 at 9:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.