0
$\begingroup$

Given the following data

        X1   X2.O      X2.Y     out
-------------------------------------
        0      1        0      0
        0      0        1      0
        1      1        0      0
        1      0        0      0
        1      1        0      0
-------------------------------------
        0      1        0      1
        1      0        0      1
        0      1        0      1
        1      0        0      1
        1      0        1      1
=====================================
Prob   0.6    0.5      0.2    0.5
P(S)   0.6    0.4      0.2    
P(-S)  0.6    0.6      0.2 
=====================================

There are two categorical variables, $x_1, x_2$ where $x_1$ has two levels and $x_2$ has three (x2 \in O, M, Y).

I would like to compute the result of naive bayes by hand to find the probability of success given x1 = 0 and x2.O = 0

I know that i have conditional independence, meaning

$$ P(A,B \vert C) = P(A \vert C) P(B \vert C) $$

I'm not sure how to calculate for this though.

edit

Here is what I've tried

I'm interested in $P(C | A, B)$ which can be written using bayes theorem as

$$ P(C | A,B) = \frac{ P(A,B | C) P(C)}{P(A,B)} $$

using conditional independence the numerator of this is

$$ P(A,B |C)P(C) = P(A | C)P(B | C) P(C) $$

and the denominator is

$$ P(A,B) = P(A,B | C)P(C) + P(A,B | ¬C)P(¬C) $$

which can be expressed as

$$ P(A,B | C)P(C) + P(A,B | ¬C)P(¬C) = P(A | C)P(B | C)P(C) + P(A | ¬C)P(B | ¬C)P(¬C) $$

giving the expression

$$ \frac{ P(A,B | C) P(C)}{P(A,B)} = \frac{ P(A | C)P(B | C) P(C) }{ P(A | C)P(B | C)P(C) + P(A | ¬C)P(B | ¬C)P(¬C)} $$

From the above table these are , using A = x1 and B= x2.O

$P(A|C) = 3/5 = 0.6$

$P(B|C) = 2/5 = 0.4$

$P(C) = 0.5$

$P(A|¬C) = 3/5 = 0.6$

$P(B|¬C) = 3/5 = 0.6$

$P(¬C) = 0.5$

giving

$$ \frac{ (0.6)(0.4)(0.5) }{ (0.6)(0.4)(0.5) + (0.6)(0.6)(0.5) } $$

Which is $0.4285$

edit 2

Is the above example a demonstration of the assumption naive bayes makes. In that there is no data in the sample set that has x1 and x2.O being true when out=1, yet we still get a probability of ~ 0.43.

$\endgroup$
  • $\begingroup$ @Tim i've edited the post with some working, thanks $\endgroup$ – baxx Apr 22 at 10:34
  • 1
    $\begingroup$ Unless I missed something, you seem to have solved it? $\endgroup$ – Tim Apr 22 at 10:46
  • $\begingroup$ @Tim perhaps... I managed to convince myself that something was incorrect about my approach, so perhaps I've been trying to find an invisible error I'm not sure. $\endgroup$ – baxx Apr 22 at 10:57
  • $\begingroup$ @Tim what i would be interested in is whether this example demonstrates the conditional independence assumption is quite a stretch - basing this on there being no instances within the data of x2.O and x1 both being true, yet we still find a probability of ~ 0.43 $\endgroup$ – baxx Apr 22 at 11:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.