I'm currently going through "Bayesian Reasoning and Machine Learning" by David Barber and it is an extremely well written and engaging book for learning the fundamentals. So a question to someone who has already done this. What are the next set of books I should go through after I have reasonable proficiency with most of the concepts in Barber?
2 Answers
$\begingroup$
$\endgroup$
1
I'd not heard of the Barber book before, but having had a quick look through it, it does look very very good.
Unless you've got a particular field you want to look into I'd suggest the following (some/many of which you've probably already heard of):
- Information theory, inference and learning algorithms, by D.J.C Mackay. A classic, and the author makes a .pdf of it available for free online, so you've no excuse.
- Pattern Recognition and Machine Learning, by C.M.Bishop. Frequently cited, though there looks to be a lot of crossover between this and the Barber book.
- Probability theory, the logic of science, by E.T.Jaynes. In some areas perhaps a bit more basic. However the explanations are excellent. I found it cleared up a couple of misunderstandings I didn't even know I had.
- Elements of Information Theory, by T.M. Cover and J.A.Thomas. Attacks probability from the perspective of, yes, you guessed it, information theory. Some very neat stuff on channel capacity and max ent. A bit different from the more bayesian stuff (I can only remember seeing one prior in the whole book).
- Statistical Learning Theory, by V.Vapnik. Thoroughly un-baysian, which may not appeal to you. Focuses on probablisitc upper bound on structural risk. Explains where support vector machines come from.
- Sir Karl Popper produced a series of works on the philosophy of scientific discovery, which feature quite a lot of stats (collections of them can be bought, but I don't have any titles to hand - apologies). Again, not bayesian in the slightest, but his discussion on falsifiability and its relationship to occams razor is (in my opinion) fascinating, and should be read by anyone involved in doing science.
-
3$\begingroup$ +1 for Jaynes, Cover and Thomas and Vapnik; after a book like Barber (or Bishop or Murphy) it is probably better to concentrate on books that go into depth of one particular idea rather than breadth. $\endgroup$ Commented May 16, 2013 at 11:08
$\begingroup$
$\endgroup$
I recently found a more computational perspective Bayesian reasoning and statistics: "Probabilistic Programming and Bayesian Methods for Hackers". This is probably equally as good an introduction to Bayesian methods as Barber.