# Naive bayes example by hand

Given the following data

        X1   X2.O      X2.Y     out
-------------------------------------
0      1        0      0
0      0        1      0
1      1        0      0
1      0        0      0
1      1        0      0
-------------------------------------
0      1        0      1
1      0        0      1
0      1        0      1
1      0        0      1
1      0        1      1
=====================================
Prob   0.6    0.5      0.2    0.5
P(S)   0.6    0.4      0.2
P(-S)  0.6    0.6      0.2
=====================================


There are two categorical variables, $$x_1, x_2$$ where $$x_1$$ has two levels and $$x_2$$ has three (x2 \in O, M, Y).

I would like to compute the result of naive bayes by hand to find the probability of success given x1 = 0 and x2.O = 0

I know that i have conditional independence, meaning

$$P(A,B \vert C) = P(A \vert C) P(B \vert C)$$

I'm not sure how to calculate for this though.

# edit

Here is what I've tried

I'm interested in $$P(C | A, B)$$ which can be written using bayes theorem as

$$P(C | A,B) = \frac{ P(A,B | C) P(C)}{P(A,B)}$$

using conditional independence the numerator of this is

$$P(A,B |C)P(C) = P(A | C)P(B | C) P(C)$$

and the denominator is

$$P(A,B) = P(A,B | C)P(C) + P(A,B | ¬C)P(¬C)$$

which can be expressed as

$$P(A,B | C)P(C) + P(A,B | ¬C)P(¬C) = P(A | C)P(B | C)P(C) + P(A | ¬C)P(B | ¬C)P(¬C)$$

giving the expression

$$\frac{ P(A,B | C) P(C)}{P(A,B)} = \frac{ P(A | C)P(B | C) P(C) }{ P(A | C)P(B | C)P(C) + P(A | ¬C)P(B | ¬C)P(¬C)}$$

From the above table these are , using A = x1 and B= x2.O

$$P(A|C) = 3/5 = 0.6$$

$$P(B|C) = 2/5 = 0.4$$

$$P(C) = 0.5$$

$$P(A|¬C) = 3/5 = 0.6$$

$$P(B|¬C) = 3/5 = 0.6$$

$$P(¬C) = 0.5$$

giving

$$\frac{ (0.6)(0.4)(0.5) }{ (0.6)(0.4)(0.5) + (0.6)(0.6)(0.5) }$$

Which is $$0.4285$$

# edit 2

Is the above example a demonstration of the assumption naive bayes makes. In that there is no data in the sample set that has x1 and x2.O being true when out=1, yet we still get a probability of ~ 0.43.

• @Tim i've edited the post with some working, thanks – baxx Apr 22 at 10:34
• Unless I missed something, you seem to have solved it? – Tim Apr 22 at 10:46
• @Tim perhaps... I managed to convince myself that something was incorrect about my approach, so perhaps I've been trying to find an invisible error I'm not sure. – baxx Apr 22 at 10:57
• @Tim what i would be interested in is whether this example demonstrates the conditional independence assumption is quite a stretch - basing this on there being no instances within the data of x2.O and x1 both being true, yet we still find a probability of ~ 0.43 – baxx Apr 22 at 11:15