0
$\begingroup$

Why is P(A) in this example 1?

In class, we were shown an illustrative example of bayes that no one including the professor could understand.

P(A|B) = P(A) * P(B|A) / P(B)

Question is:

Use Bayes Theorem to compute the probability of each hair color given that the subject has blue eyes.

Table for the question:

eye_hair = pandas.DataFrame({
    'black': [0.11, 0.03, 0.03, 0.01], 
    'brunette': [0.2, 0.14, 0.09, 0.05],
    'red': [0.04, 0.03, 0.02, 0.02],
    'blond': [0.01, 0.16, 0.02, 0.03],
}, index=['brown', 'blue', 'hazel', 'green'])
eye_hair['marginal_eye'] = eye_hair.sum(axis=1)
eye_hair.loc['marginal_hair'] = eye_hair.sum(axis=0)

print(eye_hair)

black brunette red blond marginal_eye

brown 0.11 0.20 0.04 0.01 0.36

blue 0.03 0.14 0.03 0.16 0.36

hazel 0.03 0.09 0.02 0.02 0.16

green 0.01 0.05 0.02 0.03 0.11

marg_hair 0.18 0.48 0.11 0.22 0.99

Answer given in class:

for color in eye_hair.columns[:4]:
    p = eye_hair.loc['blue', color] * 1.0 /eye_hair.loc['blue', 'marginal_eye'] 
    print('Probability of blue eyes, for Hair color of ' + color + ': ' + '%0.3f' % p)

Probability of blue eyes, for Hair color of black: 0.083

Probability of blue eyes, for Hair color of brunette: 0.389

Probability of blue eyes, for Hair color of red: 0.083

Probability of blue eyes, for Hair color of blond: 0.444

Why do we use 1.0 here? Aka why is P(A) == 1?

My professor's explanation was because it makes all the final probabilities sum to one.

Thanks!

$\endgroup$
0
$\begingroup$

The formula that your Professor is using in this code is the following:

P(A|B) = P(A & B) / P(B)

And NOT

P(A|B) = P(A) * P(B|A) / P(B)

(they are equivalent)

eye_hair.loc['blue', color] is P(blue & color) i.e., the probability that someone has blue hair and the given color. This value is divided by P(blue).

Thus, we end up with:

P(color | blue) = P(blue & color) / P(blue)

The 1.0 is used to convert the results to float, which is not really needed because the data is already in float, even if you're using python 2.* (python 3 automatically returns a float).

In short, ignore the 1.0 and if I may add, try to learn these things online instead.

$\endgroup$
0
$\begingroup$

None of the rows or columns sum to 1, but the entire table sums to 0.99. I guess there is a typo in one of the numbers.

This means that the numbers we're looking at are not the conditional probabilities, P(B|A), but the joint probabilities, P(A & B). This is because sum(P(B|A), axis=B) == 1, so they cannot be the conditionals.

They are related: P(A & B) == P(B|A) * P(A). So the real reason why using 1.0 works is that P(A) is already included in the table.

BTW, this was a question about probability, not programming, so it ought to be on https://stats.stackexchange.com rather than here.

$\endgroup$
  • $\begingroup$ "None of the rows or columns sum to 1, but the entire table sums to 0.99. I guess there is a typo in one of the numbers." They don't need to. If any row/column sums to 1, that would imply that the probability of that category is 1, which is not what is intended. The sum of sum of (rows/columns) should be 1, which is happening (0.99 is because of loss of precision). $\endgroup$ – axiom Apr 11 at 2:22
  • $\begingroup$ Yes, that was my point. OP seemed to be under the impression that the table entries were the conditional probabilities. You can see that they are the joint probabilities by this reasoning (also from the definition of the marginals). $\endgroup$ – Subhaneil Lahiri Apr 11 at 6:41
0
$\begingroup$

In your code, eye_hair.loc['blue', color] is actually the probability of both events happening simultaneously: "eye color = blue" and "hair color = color".

Mathematically, that is: P(A∩B).

Which is equal to the numerator in the right side of your Bayes theorem expression:

P(A∩B) = P(A) * P(B|A)

Your professor's explanation is right in terms of how to develop an intuition for Bayes' theorem from a practical point of view: it is basically redefining the sample space to the event you're conditioning on. So the answer to the question is just taking the probabilities for the events in your "eye color = blue" row and "re-scaling" them so they sum up to 1 (which is achieved by dividing the probability values in that row by the marginal of "eye color = blue").

Hope that helps.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.