21
$\begingroup$

In Bayes' formula:

$$P(x|a) = \frac{P(a|x) P(x)}{P(a)}$$

can the posterior probability $P(x|a)$ exceed 1?

I think it is possible if for example, assuming that $0 < P(a) < 1$, and $P(a) < P(x) < 1$, and $P(a)/P(x) < P(a|x) < 1$. But I'm not sure about this, because what would it mean for a probability to be greater than one?

Improve this question
$\endgroup$
6
  • 6
    $\begingroup$ One should be precise in defining notation. It is unclear what $P(\cdot)$ represents. If $P(\cdot)$ is (a) a probability distribution (in which case $a$ and $x$ are sets) or (b) a mass function on a discrete space, then the answers you already have are essentially correct. If $P(\cdot)$ is understood to be a density function, then it is not true that $P(x \mid a) \le 1$. The reason for nitpicking is that all three types of functions satisfy Bayes rule. The notation $P(\cdot)$ usually is for a distribution, but using lower case characters for arguments suggests a density. $\endgroup$
    – guy
    May 1, 2018 at 2:28
  • 2
    $\begingroup$ $P(x\mid a) = \frac{P(x,a) }{P(a)} \le \frac{P(a) }{P(a)} =1$ so the posterior probability cannot exceed $1$. (The posterior density is a different matter - plenty of continuous distributions have densities exceeding $1$ for some values) $\endgroup$
    – Henry
    May 1, 2018 at 10:17
  • $\begingroup$ If the calculated posterior exceeds one, you made a mistake somewhere. $\endgroup$
    – Emil M Friedman
    May 1, 2018 at 20:30
  • 2
    $\begingroup$ @EmilMFriedman, your answer is ambiguous (and, for that reason, potentially harmful), because it doesn't indicate whether it refers to a "calculated posterior" probability or density. $\endgroup$
    – whuber ♦
    May 1, 2018 at 22:21
  • $\begingroup$ The unity barrier in probability can and has been broken. See my post AT stats.stackexchange.com/questions/4220/… . $\endgroup$
    – Mark L. Stone
    May 2, 2018 at 2:13

3 Answers 3

Reset to default
9
$\begingroup$

The assumed conditions do not hold- it can never be true that $P(a)/P(x) < P(a|x)$ by the definition of conditional probability:

$P(a|x) = P(a\cap x) / P(x) \leq P(a) / P(x)$

Improve this answer
$\endgroup$
29
$\begingroup$

No, it is not possible for the posterior probability to exceed one. That would be a breach of the norming axiom of probability theory. In your question you specify that $\mathbb{P}(a)/\mathbb{P}(x) < \mathbb{P}(a | x)$ as part of your example. However, using the rules of conditional probability, you must have:

$$\mathbb{P}(a | x) = \frac{\mathbb{P}(a,x)}{\mathbb{P}(x)} \leqslant \frac{\mathbb{P}(a)}{\mathbb{P}(x)}.$$

This means that you cannot have the inequality conditions you have specified. (Incidentally, this is a good question: it is good that you are probing the probability laws looking for problems. It shows that you are exploring these matters with a greater degree of rigour than most students.)

An additional point: It is worth making one additional point about this situation, which is about the logical priority of different characteristics of probability. Remember that probability theory starts with a set of axioms that characterise what a probability measure actually is. From these axioms we can derive "rules of probability" which are theorems derived from the axioms. These rules of probability must be consistent with the axioms to be valid. If you ever found that a rule of probability leads to a contradiction with one of the axioms (e.g., the probability of the sample space is greater than one), this would not falsify the axiom - it would falsify the probability rule. Hence, even if it were the case the Bayes' rule could lead to a posterior probability greater than one (it doesn't), this wouldn't mean that you can have a posterior probability greater than one; it would simply mean that Bayes' rule is not a valid rule of probability.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Should the final numerator be P(x)? $\endgroup$
    – BallpointBen
    May 1, 2018 at 15:53
  • $\begingroup$ Still showing P(a) for me $\endgroup$
    – BallpointBen
    May 1, 2018 at 22:59
  • 1
    $\begingroup$ It is supposed to be P(a) in numerator. The inequality is showing the OP that he cannot have P(a|x) > P(a)/P(x) as he specified in his question. $\endgroup$
    – Ben
    May 1, 2018 at 23:38
10
$\begingroup$

The Bayes formula $\displaystyle P(B \mid A) = \frac{P(A\mid B)P(B)}{P(A)}$ cannot give values for $P(B\mid A)$ exceeding $1$. An intuitive way to see this is to express $P(A)$ via the law of total probability as $$P(A) = P(A\mid B)P(B) + P(A\mid B^c)P(B^c)$$ giving that $$P(B \mid A) = \frac{P(A\mid B)P(B)}{P(A)} = \frac{P(A\mid B)P(B)}{P(A\mid B)P(B) + P(A\mid B^c)P(B^c)}$$ which shows that the numerator is just one of the terms in the sum in the denominator, and so the fraction cannot exceed $1$ in value.

Improve this answer
$\endgroup$
2
  • $\begingroup$ +1 this is the easiest proof to me. $\endgroup$
    – user541686
    May 2, 2018 at 9:07
  • $\begingroup$ @Mehrdad Thanks. The other answers essentially prove that a conditional probability $P(B\mid A)$ cannot exceed $1$ via the result that $P(A\mid B)P(B) = P\A\cap B)$ cannot exceed $P(A)$ because $A\cap B \subset A$ and so it must be that $P\A\cap B) \leq P(A)$, and have little relationship per se to Bayes' formula (as it is used in statistics to derive posterior probabilities from prior probabilities). $\endgroup$
    – Dilip Sarwate
    May 2, 2018 at 19:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.