Naïve Bayes Theorem for multiple features

I understand the basic principles for the naïve bayes classification with one feature:

$$P(Class|feature) = (P(f|Class) * P(Class)) / P(f)$$

We have a dataset that has the following attributes/features:

day | outlook | temperature | humidity | wind | play

• Day is just a number (sequence)
• Outlook can be [sunny | overcast | rain]
• Temperature can be [cool | mild | hot]
• Humidity can be [normal | high]
• Wind can be [strong | weak]
• Play is [yes | no]

Now, we have a new instance: today = (sunny, cool, high, strong) and we want to know if we can play outside. This is Bayes classification with multiple features, as you've recognized.

The image below is a slide from my course at uni, however I don't understand anything of it.

Who can explain to work out the above formulas to me like I'm five, maybe with Python code? I'd like to understand how I can do naïve bayes classification for multiple features.

• There is no naive Bayes theorem, there are naive Bayes algorithm and Bayes theorem. What exactly is unclear for you?
– Tim
Jan 17, 2018 at 17:40
• @Tim It's about the Bayes Theorem with multiple features, as the image shows. I'd like an explanation of the steps I should take to do that and maybe some code :) Jan 18, 2018 at 11:58
• Your image related to naive Bayes algorithm... what exactly is unclear for you in here? Have you seen stats.stackexchange.com/questions/21822/… or stats.stackexchange.com/questions/33185/… ?
– Tim
Jan 18, 2018 at 13:16
• @Tim I understand the algorithm for one feature ( P(Class|feature) = P(feature|Class)*P(Class)/P(feature)). However, when I get multiple features (for example outlook, wind and humidity) I get lost with the formula in the image I included. I just don't understand what to do, how to tackle this. Jan 18, 2018 at 13:23
• @Tim I added more detail to my question, do you think you could help me out with the question? Jan 18, 2018 at 14:50

Naive Bayes algorithm assumes that your features are independent (hence we call it "naive", since it makes the naive assumption about independence, so we don't have to care about dependencies between them). What follows, we model

\begin{align} p(C_k, x_1, x_2, ..., x_n) &\propto p(x_1 | C_k) \, p(x_2 | C_k) \dots p(x_n | C_k) \, p(C_k) \\ &= \prod_{i=1}^n p(x_i|C_k) \, p(C_k) \end{align}

This follow from the Bayes theorem and independence. So in your example today = (sunny, cool, high, strong), you look at $p($day = sunny $|$ play = yes $)$, and $p($outlook = cool $|$ play = yes $)$, etc.

For more details see the great Wikipedia article on naive Bayes algorithm, the Understanding Naive Bayes thread on our site and the A simple explanation of Naive Bayes Classification thread on StackOverflow.com.

• Let me check if I understand correctly: P(Class) multiplied by P(feature 1|Class) x ... x ... P(feature n|Class) for every feature? Jan 18, 2018 at 15:21
• @jbehrens94 exactly, see the links for details and examples
– Tim
Jan 18, 2018 at 15:24
• Thanks, Tim! I've got just one more question, in the image in my question, there's something about estimation, how does that work? Why and when would I do that? Jan 19, 2018 at 8:28
• @jbehrens94 see the links I provided. TLDR: naive Bayes algorithm uses empirical probabilities (observed fractions) and then classifies by choosing the class that has greatest a posteriori probability (that's the argmax part). As simple as that.
– Tim
Jan 19, 2018 at 8:55