8
$\begingroup$

I was wondering if there is a subset of topics of frequentist statistics that one should know before starting to learn Bayesian statistics. Once I read that it seems that the two trends are antagonistic to each other; like for example frequentist analysis is based heavily on assumptions (hypothesis) that are made over observed data; while Bayesian statistics rely more in the construction of a prior model to infer posterior information about it.

In any case, which topics of frequentist or general statistics should I know before embarking upon Bayesian statistics?

Improve this question
$\endgroup$
1
  • $\begingroup$ "antagonist one of each other" -> "antagonistic to each other"? $\endgroup$
    – Faheem Mitha
    Mar 17, 2015 at 19:08

2 Answers 2

Reset to default
6
$\begingroup$

It is not necessary to call it frequentist material, rather material from probability and statistics in general.

Here are some examples of prior knowledge that, in my opinion, would be handy:

  1. What are densities, (conditional) distributions, expectations etc.?
  2. Some specific distributional families (Beta, normal, uniform etc.)
  3. Most likely you will want to apply Bayesian methods to real data, so statistical software. My favorite: R
  4. Some mathematics: Matrix algebra, integration, ...
  5. Also, it could be handy to be familiar with some statistical models, such as the linear model $y=X\beta+u$.
  6. Given the heavy emphasis on the likelihood, it cannot hurt to have heard about maximum likelihood before

The Bayesian paradigm being a subjective one, I am sure others will disagree with or add to this list...

Improve this answer
$\endgroup$
2
  • $\begingroup$ So your answer to the question "what frequentist topics should I learn" is basically "none"? Because you explicitly left out any frequentist topics like null hypotheses, p-Values or unbiased estimators $\endgroup$
    – nikie
    Mar 17, 2015 at 16:35
  • 1
    $\begingroup$ Well, I must say I have never met (or at least talked to about) anybody who learned Bayesian statistics before frequentist statistics, but that would be my take, yes, as indeed, these topics are relatively less important from a Bayesian point of view. Of course, there are results on Bayesian hypothesis tests, and frequentist properties of Bayes estimators (such as unbiasedness). But I would argue these are not necessary to start learning, as you asked. $\endgroup$
    – Christoph Hanck
    Mar 17, 2015 at 19:17
10
$\begingroup$

You don't have to learn 'frequentist' or Bayesian statistics in any particular order. You should first learn whatever you need to understand the findings in your field, and then you should understand the mathematical (computation) and philosophical (interpretation) relationships between the techniques. There is no teacher like real data, so that is always the first concern.

There's no particular reason you couldn't learn them at the same time. It's helpful to know the gist of calculus for Bayes, which is presumably where its reputation for being "harder" comes from, but I wouldn't call it necessary now that we have much better software than just a few years ago. If you are new to statistics and want to play around with both the frequentist and Bayesian framework, I can recommend the new JASP software. If you like R, the BayesFactor package is solid.

If you want to start from frequentism, I would suggest knowing the following:

  1. The full and exact interpretation for all of the following items.
  2. The relationship between p-values, confidence intervals, sample sizes, power and error rates.
  3. The relationship between Z-tests, t-tests, analysis of variance and linear regression.
  4. The relationship between linear regression and nonlinear regression, as well as parametric versus nonparametric tests.
  5. The relationship between dummy variables, contrasts and effects coding.
  6. The full and exact interpretation for all of the preceding items.

That sounds like a lot, but these things are all connected in fundamental ways. Every inference boils down to the same essential thing: we want to make correct predictions about unobserved data, based on a model of observed data, by comparing two or more models. We do this by computing our confidence, for some definition of "confidence," in two or more models and taking the ratio. At its most basic, that's all.

A lot of the controversy is really just about formalizing "confidence," and while it's an important discussion that I'm glad we're having, it's also not something you need to be aware of right now. In the frequentist framework, special steps are taken to create an implicit null model to put in the denominator, whereas in the Bayesian framework, both models are stated explicitly, but the actual output and interpretations for both frameworks involve a substantial degree of subjectivity. For frequentism, it's in the construction of maximum likelihood and choice of error rate, and for Bayesians, it's in the prior. Everyone should learn both, in my view.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.