# What math/stats knowledge does learning Bayesian probability require?

I study undergraduate "pure" math and philosophy. I know that a number of philosophers use Bayesian probability to augment their epistemic logic. My school teaches Bayesian probability as a brief part of a fourth year class. Enrolling in it requires completing a series of stats classes that are outside of my path. However, for a few reasons, I suspect that those prerequisites exist to prepare students to learn the other elements of the class that includes the lessons on BP, and that learning BP doesn't require a three year trek through non-Bayesian probability theory. That said, I really don't know. Perhaps I'm wrong.

What math/stats knowledge does learning Bayesian probability require?

• Bayesian probability is naturally a tiny part of a statistics course, as Bayesian statistics use standard (Kolmogorov) probability theory to construct posterior distributions and derive optimal decision procedures. The place to find such a course is outside statistics and probability, most likely in philosophy. Dec 23, 2014 at 15:14
• I think that you would get the most definitive answer to this question by taking a look at the book/syllabus of the course, and seeing if you understand the material given your current background. Speaking personally, I think it's a mistake to ignore stated prerequisites. They exist for a reason. Moreover, even if the part of Bayesian probability is small, ignoring the rest of the course just because you wish to learn solely about Bayesian probability is a huge mistake, in my opinion. I think this narrow mindedness is the very definition of why a little learning is a dangerous thing. Dec 23, 2014 at 15:16
• @rocinante I agree. Almost everything in the statistics stream is near the top of my bucket list of things to learn in life. It sounds fascinating. Presently, I'm trying to prepare myself as best I can for a competitive graduate philosophy program. I figured pure math would be the best use of the time and money I have to do that.
– Hal
Dec 23, 2014 at 15:27
• "Pure mathematicians" scare me. They often can't take integrals. Dec 23, 2014 at 18:35
• Get hold of Gelman's Bayesian Data Analysis. See how far you can reach. If you have done some calculus and linear algebra before then you may find that is quite far. It's also a very good book. Dec 23, 2014 at 20:39

You need working knowledge of calculus, like being able to take integrals, and not like knowing Weierstrass theorem. For instance, if you can take this integral without looking at any references with a pen and a paper, you're probably equipped to take the course:

$\int_{-\infty}^{\infty}\frac{1}{\sqrt{2\pi}}e^{-\frac{x^2}{2}}|x^3|dx$

Knowing linear algebra helps too, but you can pick it up on the way. I'm not talking about anything crazy, it's simple matrix manipulations, pretty much within what's described in Algebra.B section of Madelung's Die Mathematischen Hilfsmittel des Physikers book. It's available online here, and is an awesome little book on applied math. For instance, if you can solve this equation, you're good to go:

$\det\left|\begin{matrix}1-\lambda& 2\\2 &1-\lambda\end{matrix}\right|=0$

You don't need measure theory and real analysis, but it helps to know them. The key is not to enroll in courses taught at math dept for math majors. Take a course specifically designed for applied folks, maybe for psychologists or other math-challenged constituents. Courses taught for physicists could be a good compromise: they have enough math to actually gain useful skills, but they don't bother with proofs and other crazy stuff mathematicians are obsessed about.

• @Arksakal How does linear algebra connect to BP? It'd be motivating to know that. I took LA this semester (as a way to work around a class where passing counted as a success). I couldn't see its relevance to anything, so it was a grind. I've got another semester of it to go.
– Hal
Dec 23, 2014 at 15:38
• @Hal, you'll deal with variance-covariance matrices, hessians, matrix inversions etc. See for instance p.95 in Albert's "Bayesian Computation with R" book, (2nd Ed). In the intro BP course, they may somehow avoid these things though. You don't need a lot of linear algebra, but some basic stuff is useful. Generally, it's a safe bet to say that in any probability course there'll be linear algebra at some point, it's unavoidable. Dec 23, 2014 at 15:47
• I would like to suggest that "proofs and other crazy stuff" have genuine value even for practitioners: they teach us why things are true and, when studied carefully, show us ways to check our own work, detect our mistakes, and develop novel methods. All these skills are essential for being an independent expert in any technical field and they bring joy to those who are curious about why things work and like to understand the basis of what they do.
– whuber
Dec 23, 2014 at 19:06
• +1 To what @whuber said. For me, proofs not only serve to teach why something is true, but also to highlight the underlying assumptions/framework under which the theory is built. This is essential for true understanding, in my opinion. You can't have informed opinions about the appropriateness of Bayesian (or whatever) methods, if you have no understanding the framework used to build them. Math is not just about the mechanics of applying algorithms. Dec 23, 2014 at 20:17
• It doesn't hurt to know the proofs. The problem is that if you go with mathematicians, it may seem that you need to possess a mountain of knowledge before getting a chance to enjoy any math-related subject, and it's not true. There are a few basic tools that are necessary to get into any area of statistics and quickly become productive. This is especially true when it comes to Bayesian stuff. Dec 23, 2014 at 20:33

I was in a similar boat, having been a Math/CS double major but needing to learn a lot of Bayesian probability for work. What I'd recommend:

1. Practical ability to do and understand integrals

2. An understanding of numerical methods for approximating integrals (sampling, Monte Carlo method)

3. Knowing the R programming language is very useful - most texts I've come across do the examples in R.

4. General understanding of model evaluation - you would certainly get this in stats, but you can get it in other fields (psychology, biology, machine learning) as well.

5. There are lots of general modeling "tricks" that I didn't see studying pure math that would have been nice to know before jumping into Bayesian probability, such as regularization and parameter selection.

Since you are a math major, you probably will see real analysis anyway, but I did not really find it that helpful for this. I did not find my background in measure theory helpful either. It very well might be that the class offered at your school is built with the assumed stats background - in which case, I recommend getting a book about Bayesian statistics/probability (I found this one useful) and working through it yourself.