Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 46 down vote favorite
50

Which is the best introductory textbook for Bayesian statistics?

One book per answer, please.

share|improve this question
10  
In the replies, please explain why you are recommending a book as "the best." – whuber♦ Jan 25 '12 at 15:33
How can there be more than one answer to a question question phrased like this? – naught101 Jul 24 '12 at 5:44

25 Answers

up vote 20 down vote

My favorite is "Bayesian Data Analysis" by Gelman, et al.

share|improve this answer
11  
This is an introductory book for people who have a decent amount of statistical background already. – John Salvatier Nov 20 '10 at 23:41
3  
I started a PhD in Statistics 9 months ago and to be honest Gelman's BDA is still above me, so I wouldn't call it an introductory text! – Sean Jun 28 '12 at 21:37
1  
@Shane, can you please to this answer explaining why it's your favourite? – naught101 Jul 24 '12 at 5:45
1  
-1, because according to multiple comments and other answers, this isn't introductory. – naught101 Jul 24 '12 at 5:48
up vote 14 down vote

Another vote for Gelman et al., but a close second for me -- being of the learn-by-doing persuasion -- is Bayesian Computation with R.

share|improve this answer
2  
Agree strongly. Both great books. Start with Bayesian Computation With R, then get Gelman et al. – PeterR Jul 20 '10 at 19:12
I agree with @PeterR. – M. Tibbits Oct 5 '10 at 13:49
up vote 14 down vote

John Kruschke released a book in mid 2011 called Doing Bayesian Data Analysis: A Tutorial with R and BUGS. It is truly introductory. If you want to walk from frequentist stats into Bayes though, especially with multilevel modelling, I recommend Gelman and Hill.

John Kruschke also has a website for the book that has all the examples in the book in BUGS and JAGS. His blog on Bayesian statistics also links in with the book.

share|improve this answer
@Amir's suggestion is a duplicate of this. (The full title of the book is "Doing Bayesian Data Analysis: A Tutorial with R and BUGS".) As a truly introductory book, I've +1'd each. – Wayne Mar 12 '12 at 21:10
updated the title and added a couple of related links. – Jeromy Anglim Mar 21 '12 at 22:49
I also vote for Kruschke's book. I've browsed most of the books listed in the answers and this is the one I found most clear. IMO, it is the most clear stats book I have read. It helps a lot that R code is available to match formulas with code. The author starts with very simple examples and builds on them. Very little background is needed. All reviews on Amazon are highly favorable. Hoff's book is my second favorite. – julieth Jul 29 '12 at 13:02
up vote 10 down vote

Sivia and Skilling, Data analysis: a Bayesian tutorial (2ed) 2006 246p 0198568320 books.goo:

Statistics lectures have been a source of much bewilderment and frustration for generations of students. This book attempts to remedy the situation by expounding a logical and unified approach to the whole subject of data analysis. This text is intended as a tutorial guide for senior undergraduates and research students in science and engineering ...

I don't know the other recommendations though.

share|improve this answer
This book it excellent. It's short and practical. – John Salvatier Nov 19 '10 at 1:48
I think this is a much better introductory text than Gelman. – Sean Jun 28 '12 at 21:38
up vote 7 down vote

The Gelman books are all excellent but not necessarily introductory in that they assume that you know some statistics already. Therefore they are an introduction to the Bayesian way of doing statistics rather than to statistics in general. I would still give them the thumbs up, however.

As an introductory statistics/econometrics book which takes a Bayesian perspective, I would recommend Gary Koop's Bayesian Econometrics.

share|improve this answer
up vote 5 down vote

Coming from non-statistical background I found Introduction to Applied Bayesian Statistics and Estimation for Social Scientists quite informative and easy to follow.

share|improve this answer
up vote 5 down vote

"Bayesian Core: A Practical Approach to Computational Bayesian Statistics" by Marin and Robert, Springer-Verlag (2007)

share|improve this answer
Thanks for mentioning our book!!! – Xi'an Jan 22 '12 at 21:14
1  
@Xi'an and gappy, please explain why this book can be recommended. For whom is it suitable? In what sense is it "best"? – whuber♦ Jan 25 '12 at 15:34
1  
I do not want to fall into self-promotion. Bayesian Core is a self-contained entry to Bayesian inference for the most common models and to computational methods (R codes provided). It requires some background in probability theory that may be too much for some readers... (It works well with our 4th and 5th year students in France.) – Xi'an Jan 26 '12 at 14:48
up vote 5 down vote

Its focus isn't strictly on Bayesian statistics, so it lacks some methodology, but David MacKay's Information Theory, Inference, and Learning Algorithms made me intuitively grasp Bayesian statistics better than others - most do the how quite nicely, but I felt MacKay explained why better.

share|improve this answer
2  
And it is available for free download at the authors page: inference.phy.cam.ac.uk/mackay/itila/book.html – PeterR Sep 8 '10 at 12:37
1  
Like Sivia, this is very nice if you have some physics background and can be rough if not. Not a good guide to any kind of applied social statistics (for that use Gelman and Hill, or Gelman et al. above) but really great for prompting you to really think about the core issues. – conjugateprior Dec 6 '10 at 21:12
This was the first book for me many years ago. – Memming Apr 14 '12 at 22:26
up vote 5 down vote

My favourite first undergraduate text for bayesian statistics is by Bolstad, Introduction to Bayesian Statistics. If you're looking for something graduate level, this will be too elementary, but for someone who is new to statistics this is ideal.

share|improve this answer
1  
I think it's good for anyone. :) – Michelle Jan 23 '12 at 1:19
up vote 5 down vote

I have read some parts of A First Course in Bayesian Statistical Methods by Peter Hoff, and I found it easy to follow. (Example R-code is provided throughout the text)

share|improve this answer
up vote 5 down vote

I am an electrical engineer and not a statistician. I spent alot of time to go through Gelman but I don't think one can refer to gelman as introductory at all. My bayesian-guru professor from Carnegie Mellon is agree with me on this. having the minimum knowledge of statistics and R and Bugs(as the easy way to DO something with Bayesian stat) Doing Bayesian Data Analysis: A Tutorial with R and BUGS is an amazing start. you can compare all offered books easily by their book cover!

share|improve this answer
This is a duplicate of @rosser's answer above. As a truly introductory book, I've +1'd each. – Wayne Mar 12 '12 at 21:09
1  
Never thought I'd say anything nice about a book with puppies on the front and cyan section headers, but... this looks just great. – conjugateprior Mar 21 '12 at 22:57
up vote 4 down vote

I've at least glanced at most of these on this list and none are as good as the new Bayesian Ideas and Data Analysis in my opinion.

Edit: It is easy to immediately begin doing Bayesian analysis while reading this book. Not just model the mean from a Normal distribution with known variance, but actual data analysis after the first couple of chapters. All code examples and data are on the book's website. Covers a decent amount of theory but the focus is applications. Lots of examples over a wide range of models. Nice chapter on Bayesian Nonparametrics. Winbugs, R, and SAS examples. I prefer it over Doing Bayesian Data Analysis (I have both). Most of the books on here (Gelman, Robert, ...) are not introductory in my opinion and unless you have someone to talk to you will probably be left with more questions then answers. Albert's book does not cover enough material to feel comfortable analyzing data different from what is presented in the book (again my opinion).

share|improve this answer
1  
"Good" in what sense? – whuber♦ Jan 25 '12 at 15:34
Good point. Good as in best introductory Bayesian textbook. I believe it to be 'better' than Bayesian Data Analysis with R by Albert and I found Bayesian Data Analysis by Gelman et al. to not suffice as an introduction. After learning some Bayesian material however, it is a good reference. – Glen Apr 9 '12 at 14:18
@Glen: but WHY do you believe it is best? Well written? Logical layout? Good examples? Is it restricted to a particular field? – naught101 Jul 24 '12 at 5:51
@Glen, thanks for the edit, +1 – naught101 Dec 10 '12 at 10:12
up vote 3 down vote

For complete beginners, try William Briggs Breaking the Law of Averages: Real-Life Probability and Statistics in Plain English

share|improve this answer
curious that that book has no reviews on Amazon – Neil McGuigan Jul 19 '10 at 23:54
up vote 3 down vote

If you're looking for an elementary text, i.e. one that doesn't have a calculus prerequisite, there's Don Berry's Statistics: A Bayesian Perspective.

share|improve this answer
up vote 2 down vote

I simply must to include MCMC in Practice. It provides an excellent introduction to MCMC, perhaps not as general as other books mentioned, but excellent for gaining insight and intuition. I would recommend reading it after (or in parallel with) Bayesian Computation with R.

share|improve this answer
Mcmc should not be the focus of an introduction to bayesian statistics, in my opinion. I think rejection sampling is more appealling as a way to understand how bayesian learning works. Also, least squares is bayesian (as is maximum likehood), so it too represents a gentler introduction to bayesian statistics, compared to mcmc. – probabilityislogic Jan 23 '12 at 9:59
My view is that mcmc should be avoided and used as a last resort - it simply takes too long in most cases (though i deal with big data sets where everything is basically mle). mcmc is a "sledge hammer" to some degree. Also mcmc is an algorithm for numerical integration. Nothing more, nothing less. It should receive the same introductory treatment as other algorithms, such as the laplace method and quadratre. Otherwise people will develop a narrow view of what "bayesian statistics" is. – probabilityislogic Jan 23 '12 at 12:13
up vote 2 down vote

Take a look at "The Bayesian Choice". It has the full package: foundations, applications and computation. Clearly written.

share|improve this answer
up vote 2 down vote

I thoroughly recommend the entertaining polemic "Probability Theory: The Logic of Science" by E.T. Jaynes.

This is an introductory text in the sense of not requiring (and in fact preferring) no previous knowledge of statistics, but it does eventually employ fairly sophisticated mathematics. Compared to most of the other answers provided, this book is not nearly as practical or easy to digest, rather it provides the philosophical bedrock to why you would want to employ Bayesian methods, and why not to use frequentist approaches. It is introductory in a historical and philosophical, but not pedagogical way.

share|improve this answer
2  
This is a brilliant book about Bayesian thinking rather than applying Bayesian methods. I think this is a good companion text to something which goes more into how do Bayesian computations. – probabilityislogic Jul 24 '12 at 12:29
That's a good way of putting it. I think Sivia and Skilling is an ideal companion text for introducing the methods in practice (which has already been suggested in another answer). – Bogdanovist Jul 24 '12 at 22:22
up vote 1 down vote

Bayesian Statistics for Social Scientists. Phillips, Lawrence D. (1973), Thomas Crowell & Co. It's very clear, very accessible, assumes no statistics knowledge, and, unlike Bolstad which I found dry, has some personality.

share|improve this answer
up vote 1 down vote

If you happen to come from the physical sciencies (physics/astronomy) I would recommend you Bayesian Logical Data Analysis for the Physical Sciences: A Comparative Approach with Mathematica® Support by Gregory (2006).

Although the "with Mathematica® Support" part of the title is there only for commercial issues (the usages of Mathematica code are very poor), the good thing about this book is that it is truly an introduction to the subject of probabilities and statistics. It even has some chapters on frequentist statistics. However, once you give it a shot, go for the book of Gelman et. al that a lot of people recommended you. Most of the material in the book of Gregory is taken lightly (if not, it wouldn't be an introduction): Gelman's book has been a truly re-awakening from Gregory's for me.

share|improve this answer
up vote 0 down vote

Gelman et al. is well-regarded but explictly intended for a graduate course. If you don't have substantial prior coursework in statistics, it is largely a waste.

share|improve this answer
2  
+0, this would serve better as a comment to @Shane's answer. You might edit your answer to recommend a book that would better sever the stated purpose for those without "substantial prior coursework in statistics". – gung Jan 23 '12 at 3:45
up vote 0 down vote

This book suggests it is aimed at entry level undergraduate level

Biostatistics: A Bayesian Introduction. By George G Woodsworth.

Published by John Wiley & Sons

share|improve this answer
up vote 0 down vote

I found an excellent introduction in Gelman and Hill (2006) Data Analysis Using Regression and Multilevel/Hierarchical Models. (Other comments mention it, but it deserves to get upvoted on its own.)

share|improve this answer
up vote 0 down vote

Not strictly Bayesian Statistics as such, but I can strongly recommend "A First Course on Machine Learning" by Rogers and Girolami, which is essentially an introduction to Bayesian approaches to machine learning. Its very well structured and clear and aimed at students without a strong mathematical background. This means it is a pretty good first introduction to Bayesian ideas. There is also MATLAB/OCTAVE code which is a nice feature.

share|improve this answer
up vote -6 down vote

This might be worth a look for people working on certain types of problems:

Millán, E. et al., 2010. Bayesian Networks for Student Model Engineering. Computers & Education. Available at: http://dx.doi.org/10.1016/j.compedu.2010.07.010.

Abstract: Bayesian networks are a graphical modeling tool that has been proven very powerful in a variety of application contexts. The purpose of this paper is to provide education practitioners with the background and examples needed to understand Bayesian netwrosk and use them to design and implement student models. The student model is the key component of any adaptive tutoring system, as it stores all the information about the student (for example, knowledge, interest, learning styles, etc.) so the tutoring system can use this information to provide personalized instruction. Basic and advanced concepts and techniques are introduced and applied in the context of typical student modeling problems. A repertoire of models of varying complexity is discussed. To illustrate the proposed methodology a Bayesian Student Model for the Simplex algorithm is implemented.

share|improve this answer
2  
-1: question's aim was to get a list of text books, not info on obscure papers about specialised sub-fields. – fabians Dec 6 '10 at 9:17

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.