Coming from a rigorous background in analysis and modern probability theory, I find Bayesian statistics straightforward and easy to understand, and frequentist statistics incredibly confusing and unintuitive. It seems that frequentists are really doing bayesian statistics, except with "secret priors" that aren't well motivated or carefully defined.

On the other hand, a lot of great statisticians who understand both perspectives ascribe to the frequentist perspective, so there must be something there that I just don't understand. Rather than giving up and declaring myself a Bayesian, I'd like to learn more about the frequentist perspective to try to really "grok" it.

What are some good references for learning frequentist statistics from a rigorous perspective? Ideally I'm looking for definition-theorem-proof type books, or perhaps hard problem sets that, by solving them, I would gain the right mindset. I've read a lot of the more "philosophical stuff" one might find searching the internet - wiki pages, random pdfs from .edu/~randomprof sites, etc - and it hasn't helped.

  • $\begingroup$ I was exactly like you ! Solid background in probability theory, but ignorant in statistics. And I was charmed by Bayesian statistics (especially Christian Robert's book). I learned frequentist statistics in Fourdrinier's book amazon.fr/… but I'm not sure you read French. Please let me note you're wrong about "secret priors". $\endgroup$ – Stéphane Laurent Sep 27 '12 at 7:10
  • $\begingroup$ This is a very wide topic and it is important to understand the difference in the interpretation of the parameters. Given that you have a strong theoretical background, it will be easy for you to understand that, in the Bayesian paradigm, a parameter is a random variable while, in frequentist statistics, a parameter is a variable/number to be estimated. Therefore, there is nothing like frequentists are using "secret priors". You can find some references here. $\endgroup$ – user10525 Sep 27 '12 at 10:57

For your background, I would start out with: http://www.amazon.com/Essentials-Statistical-Inference-Probabilistic-Mathematics/dp/0521548667/ref=sr_1_1?s=books&ie=UTF8&qid=1348728521&sr=1-1&keywords=essentials+of+statistical+inference

which is short and reasonably complete. The preface says it is written for a first intro to math stat for oxford 4th year math students. It also includes some very modern ideas.

But you also need something more conceptual, and you cannot find better than Sir David Cox to teach this: D R Cox: "Principles of Statistical Inference" Cambridge UP 2006. This is very rigourous, but in a statistical, not mathematical sense. This is about the concepts, about the Why's and not the How's!

  • $\begingroup$ I think he could also look at some of von Mises' writings. The classic by Cramer on mathematical statistics is certainly data but gets to the foundational things that haven't changed much since the 1940s. I can understand how Bayesian methods can sound intuitive but prectical implimentation is not as clear inspite of the MCMC revolution. $\endgroup$ – Michael R. Chernick Sep 27 '12 at 20:49
  • $\begingroup$ Also statements like "It seems that frequentists are really doing bayesian statistics, except with "secret priors" that aren't well motivated or carefully defined." maybe show that the OP really does need to get a better understanding of the foundations of statistics. Concepts like confidence intervals and p-values may be hard to understand but that doesn't make them wrong. If you are going to do serious statistics it may be worthwhile to make the effort to understand these concepts. $\endgroup$ – Michael R. Chernick Sep 27 '12 at 20:53
  • $\begingroup$ The frequentist idea that probabilities can be defined in terms of long run frequencies seem very intuitive to me. If you want to know whether or not you are flipping a fair coin doesn't it make erfect sense that if you toss it 10,000 times and get close to 5000 heads that it is indicating that the coin is fair (i.e. the probability of a head is 1/2). $\endgroup$ – Michael R. Chernick Sep 27 '12 at 20:56
  • $\begingroup$ @kjetil Thank you for the references. I browsed through these books at the library and they looked good so I bought them. $\endgroup$ – Nick Alger Sep 30 '12 at 3:23
  • $\begingroup$ @MichaelChernick Yes you're right I don't have a very good grasp of statistics, my goal is to remedy this. That frequentist idea is actually not intuitive to me at all.. :/ I was hoping if it is presented in full rigor with $\forall$'s and $\epsilon$'s and functions between sets and such, then I could make sense of it. Bayesian statistics is much easier since I can just think about conditional expectation of random variables. $\endgroup$ – Nick Alger Sep 30 '12 at 3:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.