# Study path to Bayesian thinking?

I am six years into a business role and have a bachelor's in physics and applied math/stats. Sean Carroll's (Caltech physicist) "The Big Picture" opened me to the idea that Bayesian statistics is one useful way of thinking about anything - inevitably you hold a prior and you should update your credence as additional information becomes available.

Is there a path to training your intuition to thinking this way? Critically, it would require repeated practice with verifiable answers through either a course, or self study that includes many problems and solutions. I do not believe simply reading will do.

Possible resources, having read every related question on this site I could find:

• "Probability Theory" by Jaynes. Pro: analytic; intuitive explanation of bayesian statistics. Con: prerequisites; missing problems/solutions.
• "Doing Bayesian Data Analysis" by Kruschke. Pro: includes problems & solutions; requires only "algebra and rusty calculus". Con: works in R, which I think provides for less intuitive learning than the analytical (I may be wrong).

If it is a multi-year path I need to take, starting elsewhere, I am happy to do so! Ideally, I would avoid the frequentist methods, as I have no use for them.

My goal is not to be a scientist, but to leverage insight into how reality works to go above and beyond the established thinking in business.

Many thanks for any suggestions!

• Think Bayes was a good starting point for me (being a programmer) you might want to have a look at it. – Mayur Kulkarni Jan 28 '18 at 5:08
• I would personally recommend Statistical Rethinking by Richard McElreath as a great way to get started with Bayesian statistics. It explains the ideas behind the methods, but also goes into practical applications and modeling. In particular, it uses R and Stan to perform the Bayesian methods. – Maurits M Jan 29 '18 at 8:20

I've started down my own path towards understanding the bayesian way of thinking and I'll share my perspective. I started off reading classic papers on the various samplers and going through derivations for the conjugate cases, and I don't think that got me very far. True, the elites are going to write their own samplers and exploit every conjugate opportunity possible. But if you want to get a good feel for the approach and potentially gain a few useful methods, there are more direct ways.

My recommendation is to find a good bayesian modeling tool that takes care of the sampling and lets you focus on specifying the likelihoods and priors. For me, this has been Stan. It's based on a particular sampler that doesn't require much tinkering. The User's Guide and Reference Manual (available on the Documentation page) reads like a textbook, and you can learn a lot by going through the examples. When you have an idea for a new model, you can try it out and usually get something working without too much time. You can see some of my own experimentation here.

We are in an age where the focus is on managing computations on enormous data sets, and software like Stan is going to encourage you to perform intense computation on even small data sets (depending on the model). But I think it's worth the time to study and understand. There are still plenty of "small data" problems out there, and it's nice to be able to frame ideas in machine learning (e.g., L2 regularization) in the bayesian context (where there is actually theory!).

From your business point of view you might be motivated by Bayesian decision theory, which is a way to apply Bayesian inference to make decisions under uncertainty.

If so, you'd find the topics that introductions to Bayesian analysis often focus on (such as specifying various prior and likelihood distributions and performing sampling computations or derivations analytically) are simply means to this ultimate end.

Here are some resources specifically on this topic:

• These are all excellent suggestions. I documented my path to Bayes at fharrell.com/post/journey which has a link to several learning resources including McElreath's online lectures. – Frank Harrell Jan 30 '18 at 12:33

I had a course on Bayesian Data Analysis in the last semester. It assumes no previous background. Here's the course homepage where the instructor has put all of the materials: https://michael-franke.github.io/BDACM_2017/

We used the Kruschke textbook for the course. It worked out fine. I don't think there's much problem with working in R. You still get to understand how things work.