2
$\begingroup$

In Bayes' theorem, what kinds of situations would lead us to chose the alternate form (#2) over the basic form (#1)?

  1. $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$
  2. $P(A|B) = \frac{P(B|A)P(A)}{P(B|A)P(A)+P(B|\lnot A)P(\lnot A)} $
$\endgroup$
  • 2
    $\begingroup$ There's no appreciable difference between the two, because $P(B)=P(B|A)P(A)+P(B|\lnot A)P(\lnot A)$. If you are given $P(B|A), P(A), P(B|\lnot A)$ and $P(\lnot A)$, you know $P(B)$ by the same reasoning. $\endgroup$ – Sycorax says Reinstate Monica Jan 5 '17 at 16:53
  • $\begingroup$ Mathematically there is no difference. But I think that by Splitting P(B) into P(B and A)+P(B and A compliment) remind us of the two disjoint sets that constitutes the event B and and then uses the identity P(B and A)=P(B|A) P(A).. This second probability manipulation of the denominator expresses it as the numerator plus an additional term. $\endgroup$ – Michael R. Chernick Jan 5 '17 at 17:14
  • $\begingroup$ You might find this answer of mine to a recent question to be interesting. $\endgroup$ – Dilip Sarwate Jan 7 '17 at 15:52
3
$\begingroup$

By the law of total probability

$$ P(B)=\sum_i P(B\mid A_i) \,P(A_i) $$

so both forms are equivalent. This is also noted in Wikipedia that provides multiple examples and goes into more detail.

Notice also that in Bayesian statistics scenario, where $f_\Theta$ is a prior distribution of some parameters of the distribution of your data $f_{X|\Theta}$ (i.e. the likelihood), then $f_X$ is a normalizing constant that is needed so that the posterior distribution integrates to unity (to be a valid distribution). In such case it may be more clear to write the normalizing constant in the extended form $\int\, f_{X|\Theta}(x|\theta)\,f_\Theta(\theta)\,d\theta$, so that it's role and origin is clear in the formula.

$$ f_{\Theta|X}(\theta| x) = \frac{f_{X|\Theta}(x|\theta)\,f_\Theta(\theta)}{f_X(x)} = \frac{f_{X|\Theta}(x|\theta)\,f_\Theta(\theta)}{\int\, f_{X|\Theta}(x|\theta)\,f_\Theta(\theta)\,d\theta} $$

$\endgroup$
1
$\begingroup$

They are identical formulas for reasons already pointed out. It's often the case in homework problems that different information is given to you. Sometimes the problem will explicitly give you $P(B)$ and sometimes it will give you $P(B|A)$ and $P(B|\lnot A)$. It's useful to think of Bayes theorem in these two identical ways so that you can quickly do these homework problems.

For example, let $A=1$ if you have some disease and $A=0$ otherwise. Let $B=1$ if a medical device says you have that disease and $B=0$ otherwise. This device is not perfect, and will incorrectly tell you that you have the disease with probability 0.10 and will correctly tell you that you have the disease with probability 0.90. This disease is not common and is seen in only 1 out of 1,000 people.

In this problem you have

  • $P(A=1) = 1/1000$
  • $P(B=1|A=1) = 0.90$
  • $P(B=1|A=0) = 0.10$

Knowing Bayes theorem in the form of your version 2 quickly allows you to calculate whether or not you have the disease given the device says you have it, or $P(A=1|B=1)$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.