18
$\begingroup$

In Bayes' formula:

$$P(x|a) = \frac{P(a|x) P(x)}{P(a)}$$

can the posterior probability $P(x|a)$ exceed 1?

I think it is possible if for example, assuming that $0 < P(a) < 1$, and $P(a) < P(x) < 1$, and $P(a)/P(x) < P(a|x) < 1$. But I'm not sure about this, because what would it mean for a probability to be greater than one?

$\endgroup$
  • 6
    $\begingroup$ One should be precise in defining notation. It is unclear what $P(\cdot)$ represents. If $P(\cdot)$ is (a) a probability distribution (in which case $a$ and $x$ are sets) or (b) a mass function on a discrete space, then the answers you already have are essentially correct. If $P(\cdot)$ is understood to be a density function, then it is not true that $P(x \mid a) \le 1$. The reason for nitpicking is that all three types of functions satisfy Bayes rule. The notation $P(\cdot)$ usually is for a distribution, but using lower case characters for arguments suggests a density. $\endgroup$ – guy May 1 '18 at 2:28
  • 2
    $\begingroup$ $P(x\mid a) = \frac{P(x,a) }{P(a)} \le \frac{P(a) }{P(a)} =1$ so the posterior probability cannot exceed $1$. (The posterior density is a different matter - plenty of continuous distributions have densities exceeding $1$ for some values) $\endgroup$ – Henry May 1 '18 at 10:17
  • $\begingroup$ If the calculated posterior exceeds one, you made a mistake somewhere. $\endgroup$ – Emil M Friedman May 1 '18 at 20:30
  • 2
    $\begingroup$ @EmilMFriedman, your answer is ambiguous (and, for that reason, potentially harmful), because it doesn't indicate whether it refers to a "calculated posterior" probability or density. $\endgroup$ – whuber May 1 '18 at 22:21
  • $\begingroup$ The unity barrier in probability can and has been broken. See my post AT stats.stackexchange.com/questions/4220/… . $\endgroup$ – Mark L. Stone May 2 '18 at 2:13
8
$\begingroup$

The assumed conditions do not hold- it can never be true that $P(a)/P(x) < P(a|x)$ by the definition of conditional probability:

$P(a|x) = P(a\cap x) / P(x) \leq P(a) / P(x)$

$\endgroup$
28
$\begingroup$

No, it is not possible for the posterior probability to exceed one. That would be a breach of the norming axiom of probability theory. Using the rules of conditional probability, you must have:

$$\mathbb{P}(a | x) = \frac{\mathbb{P}(a,x)}{\mathbb{P}(x)} \leqslant \frac{\mathbb{P}(a)}{\mathbb{P}(x)}.$$

This means that you cannot have the inequality conditions you have specified. (Incidentally, this is a good question: it is good that you are probing the probability laws looking for problems. It shows that you are exploring these matters with a greater degree of rigour than most students.)

An additional point: It is worth making one additional point about this situation, which is about the logical priority of different characteristics of probability. Remember that probability theory starts with a set of axioms that characterise what a probability measure actually is. From these axioms we can derive "rules of probability" which are theorems derived from the axioms. These rules of probability must be consistent with the axioms to be valid. If you ever found that a rule of probability leads to a contradiction with one of the axioms (e.g., the probability of the sample space is greater than one), this would not falsify the axiom - it would falsify the probability rule. Hence, even if it were the case the Bayes' rule could lead to a posterior probability greater than one (it doesn't), this wouldn't mean that you can have a posterior probability greater than one; it would simply mean that Bayes' rule is not a valid rule of probability.

$\endgroup$
  • 1
    $\begingroup$ Should the final numerator be P(x)? $\endgroup$ – BallpointBen May 1 '18 at 15:53
  • $\begingroup$ Still showing P(a) for me $\endgroup$ – BallpointBen May 1 '18 at 22:59
  • 1
    $\begingroup$ It is supposed to be P(a) in numerator. The inequality is showing the OP that he cannot have P(a|x) > P(a)/P(x) as he specified in his question. $\endgroup$ – Reinstate Monica May 1 '18 at 23:38
9
$\begingroup$

The Bayes formula $\displaystyle P(B \mid A) = \frac{P(A\mid B)P(B)}{P(A)}$ cannot give values for $P(B\mid A)$ exceeding $1$. An intuitive way to see this is to express $P(A)$ via the law of total probability as $$P(A) = P(A\mid B)P(B) + P(A\mid B^c)P(B^c)$$ giving that $$P(B \mid A) = \frac{P(A\mid B)P(B)}{P(A)} = \frac{P(A\mid B)P(B)}{P(A\mid B)P(B) + P(A\mid B^c)P(B^c)}$$ which shows that the numerator is just one of the terms in the sum in the denominator, and so the fraction cannot exceed $1$ in value.

$\endgroup$
  • $\begingroup$ +1 this is the easiest proof to me. $\endgroup$ – Mehrdad May 2 '18 at 9:07
  • $\begingroup$ @Mehrdad Thanks. The other answers essentially prove that a conditional probability $P(B\mid A)$ cannot exceed $1$ via the result that $P(A\mid B)P(B) = P\A\cap B)$ cannot exceed $P(A)$ because $A\cap B \subset A$ and so it must be that $P\A\cap B) \leq P(A)$, and have little relationship per se to Bayes' formula (as it is used in statistics to derive posterior probabilities from prior probabilities). $\endgroup$ – Dilip Sarwate May 2 '18 at 19:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.