7
$\begingroup$

I know the Bayes rule is derived from the conditional probability. But intuitively, what the difference? The equation looks the same to me. The nominator is the joint probability and the denominator is the probability of the given outcome.

This is the conditional probability: $P(A∣B)=\frac{P(A \cap B)}{P(B)}$

This is the Bayes' rule: $P(A∣B)=\frac{P(B|A) * P(A)}{P(B)}$.

Isn't $P(B|A)*P(A)$ and $P(A \cap B)$ the same? When $A$ and $B$ are independent, there is no need to use the Bayes rule, right?

$\endgroup$
  • 2
    $\begingroup$ If you would add the specific equations that look the same to you to your question, someone might be able to help you. The two that I am familiar with look quite different to me but there is a long tradition on stats.SE to say that Bayes formula is $$P(A\mid B) = \frac{P(A\cap B)}{P(B)}$$ which is actually the definition of the conditional probability of $A$ given $B$, and not Bayes formula at all. $\endgroup$ – Dilip Sarwate Dec 9 '16 at 2:59
  • $\begingroup$ @DilipSarwate, I have updated my question. $\endgroup$ – alphabetagamma Dec 9 '16 at 18:36
  • $\begingroup$ To your final question: yes these are the same! That doesn't mean Bayes' rule isn't a useful formula, however. The conditional probability formula doesn't give us the probability of A given B. Semantically, I'd say there's always a need to use Bayes' rule, but when A and B are independent the rule can be reduced to a much simpler form. $\endgroup$ – Jacob Socolar Dec 9 '16 at 19:03
  • $\begingroup$ I understand Bayes rule is useful. Given A and B are not independent, what the difference of conditional probability function and Bayes rule if the nominators are basically the same( correct me if i am wrong)? $\endgroup$ – alphabetagamma Dec 9 '16 at 19:11
  • $\begingroup$ My answer here provides another view of essentially this issue. $\endgroup$ – GeoMatt22 Dec 10 '16 at 5:08
16
$\begingroup$

OK, now that you have updated your question to include the two formulas:

$$P(A\mid B) = \frac{P(A\cap B)}{P(B)} ~~ \text{provided that } P(B) > 0, \tag{1}$$ is the definition of the conditional probability of $A$ given that $B$ occurred. Similarly, $$P(B\mid A) = \frac{P(B\cap A)}{P(A)} = \frac{P(A\cap B)}{P(A)} =~~ \text{provided that } P(A) > 0, \tag{2}$$ is the definition of the conditional probability of $B$ given that $A$ occurred. Now, it is true that it is a trivial matter to substitute the value of $P(A\cap B)$ from $(2)$ into $(1)$ to arrive at $$P(A\mid B) = \frac{P(B\mid A)P(A)}{P(B)} ~~ \text{provided that } P(A), P(B) > 0, \tag{3}$$ which is Bayes' formula but notice that Bayes's formula actually connects two different conditional probabilities $P(A\mid B)$ and $P(B\mid A)$, and is essentially a formula for "turning the conditioning around". The Reverend Thomas Bayes referred to this in terms of "inverse probability" and even today, there is vigorous debate as to whether statistical inference should be based on $P(B\mid A)$ or the inverse probability (called the a posteriori or posterior probability).

It is undoubtedly as galling to you as it was to me when I first discovered that Bayes' formula was just a trivial substitution of $(2)$ into $(1)$. Perhaps if you have been born 250 years ago, you could have made the substitution and then people today would be talking about the AlphaBetaGamma formula and the AlphaBetaGammian heresy and the Naive AlphaBetaGamma method$^*$ instead of invoking Bayes' name everywhere. So let me console you on your loss of fame by pointing out a different version of Bayes' formula. The Law of Total Probability says that $$P(B) = P(B\mid A)P(A) + P(B\mid A^c)P(A^c) \tag{4}$$ and using this, we can write $(3)$ as

$$P(A\mid B) = \frac{P(B\mid A)P(A)}{P(B\mid A)P(A) + P(B\mid A^c)P(A^c)}, \tag{5}$$ or more generally as $$P(A_i\mid B) = \frac{P(B\mid A_i)P(A_i)}{P(B\mid A_1)P(A_1) + P(B\mid A_2)P(A_2) + \cdots + P(B\mid A_n)P(A_n)}, \tag{6}$$ where the posterior probability of a possible "cause" $A_i$ of a "datum" $B$ is related to $P(B\mid A_i)$, the likelihood of the observation $B$ when $A_i$ is the true hypothesis and $P(A_i)$, the prior probability (horrors!) of the hypothesis $A_i$.


$^*$ There is a famous paper R. Alpher, H. Bethe, and G. Gamow, "The Origin of Chemical Elements", Physical Review, April 1, 1948, that is commonly referred to as the $\alpha\beta\gamma$ paper.

$\endgroup$
  • 1
    $\begingroup$ Hello Sir, Could you please explain what you mean by 'turning the conditioning around'? $\endgroup$ – formatkaka Feb 18 '17 at 17:53
  • 1
    $\begingroup$ @Siddhant Going from $P(A\mid B)$ to $P(B\mid A)$ is what I mean by "turning the conditioning around". Please ignore the phrase, which I made up on the spot to give a name to what Bayes' Theorem does (it gives an expression for $P(A\mid B)$ in terms of $P(B\mid A)$) since it confuses you so much. $\endgroup$ – Dilip Sarwate Feb 18 '17 at 20:52
1
$\begingroup$

Difference between conditional probability and bayes theorem....

In Conditional probability we find the probability of an event given that some event has already occurred ... but In Bayes theorem, we find just the opposite ... ie we find the cause of some event that has already occurred..

Let's take an example... lets say there are three vehicles CAR, TRUCK and BIKE... One of the vehicles meets with an accident ... Acc to Bayes theorem.. What is the probability that the vehicle was a car given that accident has already occurred.. Here ... we see that the accident has already occurred and we are finding the probability of cause of that accident.. As far as Conditional probability is concerned.... we find the probability of, occurrence of an accident given that it happened due to car

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.