Naïve Bayes Theorem for multiple features

Question

I understand the basic principles for the naïve bayes classification with one feature:

$$ P(Class|feature) = (P(f|Class) * P(Class)) / P(f) $$

We have a dataset that has the following attributes/features:

day | outlook | temperature | humidity | wind | play

• Day is just a number (sequence)
• Outlook can be [sunny | overcast | rain]
• Temperature can be [cool | mild | hot]
• Humidity can be [normal | high]
• Wind can be [strong | weak]
• Play is [yes | no]

Now, we have a new instance: today = (sunny, cool, high, strong) and we want to know if we can play outside. This is Bayes classification with multiple features, as you've recognized.

The image below is a slide from my course at uni, however I don't understand anything of it.

Who can explain to work out the above formulas to me like I'm five, maybe with Python code? I'd like to understand how I can do naïve bayes classification for multiple features.

Answer 1

Naive Bayes algorithm assumes that your features are independent (hence we call it "naive", since it makes the naive assumption about independence, so we don't have to care about dependencies between them). What follows, we model

$$ \begin{align} p(C_k, x_1, x_2, ..., x_n) &\propto p(x_1 | C_k) \, p(x_2 | C_k) \dots p(x_n | C_k) \, p(C_k) \\ &= \prod_{i=1}^n p(x_i|C_k) \, p(C_k) \end{align} $$

This follow from the Bayes theorem and independence. So in your example today = (sunny, cool, high, strong), you look at $p($day = sunny $|$ play = yes $)$, and $p($outlook = cool $|$ play = yes $)$, etc.

For more details see the great Wikipedia article on naive Bayes algorithm, the Understanding Naive Bayes thread on our site and the A simple explanation of Naive Bayes Classification thread on StackOverflow.com.