13
$\begingroup$

Coming from a rigorous background in analysis and modern probability theory, I find Bayesian statistics straightforward and easy to understand, and frequentist statistics incredibly confusing and unintuitive. It seems that frequentists are really doing bayesian statistics, except with "secret priors" that aren't well motivated or carefully defined.

On the other hand, a lot of great statisticians who understand both perspectives ascribe to the frequentist perspective, so there must be something there that I just don't understand. Rather than giving up and declaring myself a Bayesian, I'd like to learn more about the frequentist perspective to try to really "grok" it.

What are some good references for learning frequentist statistics from a rigorous perspective? Ideally I'm looking for definition-theorem-proof type books, or perhaps hard problem sets that, by solving them, I would gain the right mindset. I've read a lot of the more "philosophical stuff" one might find searching the internet - wiki pages, random pdfs from .edu/~randomprof sites, etc - and it hasn't helped.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ I was exactly like you ! Solid background in probability theory, but ignorant in statistics. And I was charmed by Bayesian statistics (especially Christian Robert's book). I learned frequentist statistics in Fourdrinier's book amazon.fr/… but I'm not sure you read French. Please let me note you're wrong about "secret priors". $\endgroup$
    – Stéphane Laurent
    Sep 27, 2012 at 7:10
  • 1
    $\begingroup$ This is a very wide topic and it is important to understand the difference in the interpretation of the parameters. Given that you have a strong theoretical background, it will be easy for you to understand that, in the Bayesian paradigm, a parameter is a random variable while, in frequentist statistics, a parameter is a variable/number to be estimated. Therefore, there is nothing like frequentists are using "secret priors". You can find some references here. $\endgroup$
    – user10525
    Sep 27, 2012 at 10:57

1 Answer 1

Reset to default
4
$\begingroup$

For your background, I would start out with: Essentials of Statistical Inference, which is short and reasonably complete. The preface says it is written for a first intro to math stat for oxford 4th year math students. It also includes some very modern ideas.

But you also need something more conceptual, and you cannot find better than Sir David Cox to teach this: D R Cox: "Principles of Statistical Inference" Cambridge UP 2006. This is very rigorous, but in a statistical, not mathematical sense. This is about the concepts, about the Why's and not the How's!

Improve this answer
$\endgroup$
7
  • 1
    $\begingroup$ I think he could also look at some of von Mises' writings. The classic by Cramer on mathematical statistics is certainly data but gets to the foundational things that haven't changed much since the 1940s. I can understand how Bayesian methods can sound intuitive but prectical implimentation is not as clear inspite of the MCMC revolution. $\endgroup$
    – Michael R. Chernick
    Sep 27, 2012 at 20:49
  • 1
    $\begingroup$ Also statements like "It seems that frequentists are really doing bayesian statistics, except with "secret priors" that aren't well motivated or carefully defined." maybe show that the OP really does need to get a better understanding of the foundations of statistics. Concepts like confidence intervals and p-values may be hard to understand but that doesn't make them wrong. If you are going to do serious statistics it may be worthwhile to make the effort to understand these concepts. $\endgroup$
    – Michael R. Chernick
    Sep 27, 2012 at 20:53
  • 1
    $\begingroup$ The frequentist idea that probabilities can be defined in terms of long run frequencies seem very intuitive to me. If you want to know whether or not you are flipping a fair coin doesn't it make erfect sense that if you toss it 10,000 times and get close to 5000 heads that it is indicating that the coin is fair (i.e. the probability of a head is 1/2). $\endgroup$
    – Michael R. Chernick
    Sep 27, 2012 at 20:56
  • $\begingroup$ @kjetil Thank you for the references. I browsed through these books at the library and they looked good so I bought them. $\endgroup$
    – Nick Alger
    Sep 30, 2012 at 3:23
  • 1
    $\begingroup$ @MichaelChernick Yes you're right I don't have a very good grasp of statistics, my goal is to remedy this. That frequentist idea is actually not intuitive to me at all.. :/ I was hoping if it is presented in full rigor with $\forall$'s and $\epsilon$'s and functions between sets and such, then I could make sense of it. Bayesian statistics is much easier since I can just think about conditional expectation of random variables. $\endgroup$
    – Nick Alger
    Sep 30, 2012 at 3:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.