I am frequentist by training and practice, but I'd like to learn more about Bayesian statistics. I know the basics, but I would be at a loss if I had to, for example, replace my normal ANOVA hypothesis testing approach with a Bayesian alternative.

What book would you recommend to learn practical Bayesian approaches? Preferably using R.


Peter D. Huff. A First Course in Bayesian Statistical Methods. Springer (2010)


Andrew Gelman et. al. Bayesian Data Analysis (3rd ed.). CRC (2013)

The Gelman book isn't constrained to R but also uses Stan, a probabilistic programming language similar to BUGS or JAGS. I believe earlier editions of the book used BUGS instead of Stan, which is probably very similar.

And finally:

John Kruschke. Doing Bayesian Data Analysis: A tutorial with R and BUGS. Academic Press (2011)

More BUGS than R, but probably the most pragmatic of the three books I've suggested. Don't let the cover deter you, this is a perfectly respectable text.

  • $\begingroup$ This is the comprehensive answer, but I wish I could accept the answer by @user777 as well. Thanks! $\endgroup$ – January Nov 14 '13 at 20:21
  • $\begingroup$ @January No worries -- I think it's more than fair that you decide what the best answer is. $\endgroup$ – Sycorax Nov 14 '13 at 20:29
  • 1
    $\begingroup$ I have to voice that the puppies on the cover of Doing Bayesian Data Analysis are respectably cute! And yes, it's a great text. $\endgroup$ – Penguin_Knight Nov 14 '13 at 22:47
  • 1
    $\begingroup$ All great texts, but very different. If you don't know the basics of Bayesian Data analysis, and isn't a math stat student I think Kruschke's text is by far the most accessible. Gelman et al is, for sure, more comprehensive but not necessarily more comprehensible depending on you background knowledge. After having read Kruschke, a great second book is the extremely pragmatical Bugs Book mrc-bsu.cam.ac.uk/bugs/thebugsbook . $\endgroup$ – Rasmus Bååth Nov 16 '13 at 21:49
  • $\begingroup$ @RasmusBååth +1, last link doesn't work. $\endgroup$ – Patrick Coulombe Feb 15 '15 at 18:24

Fortuitous timing, as Bayesian Data Analysis, 3rd ed was just released. It's a good general-purpose text, with an emphasis on hierarchical methods, a section on advanced computation (that is, Markov chain Monte Carlo), and an appendix on Gelman's Bayesian inference tool, rstan.

The text focuses on statistics rather than programming, though, so perhaps this answer does not fit your R needs. That said, I've been able to recreate the text examples in R simply based on his clear prose descriptions.

  • $\begingroup$ Aw, posted while I was editing my comment! no fair! :p $\endgroup$ – David Marx Nov 14 '13 at 19:56
  • $\begingroup$ @DavidMarx To be fair, Gelman probably should have come to mind first. ;-) $\endgroup$ – Sycorax Nov 14 '13 at 19:56
  • 2
    $\begingroup$ I had to give a shout out to the textbook used for my Bayesian class. We didn't use Gelman, but I follow his work on my own. Hell, while we're spreading the Gelman love, here's his blog in case anyone hasn't seen it. He posts quite regularly and the articles are generally pretty interesting: andrewgelman.com $\endgroup$ – David Marx Nov 14 '13 at 20:01
  • $\begingroup$ Yes, I know his blog. To what extent is this a book on the underlying mathematics, and to what extent on the everyday practice? $\endgroup$ – January Nov 14 '13 at 20:20
  • $\begingroup$ It's not written in math, if that's what you mean. All of the concepts are applied to examples from either his own or others' research. $\endgroup$ – Sycorax Nov 14 '13 at 20:27

Both are introductory, but useful imho:

Bayesian Computation With R, by Jim Albert

Applied Bayesian Statistics, With R and OpenBUGS Examples, by

  • $\begingroup$ Giving full titles and authors names could be helpful in case the links go dead in the future (and even if they don't). $\endgroup$ – Richard Hardy Aug 28 '18 at 19:11

Not the answer you're looking for? Browse other questions tagged or ask your own question.