# Given more information, can a probability lessen?

Let $A$, $B$ and $C$ be events in the same probability space. Does \begin{align} \mathbb P(A\,|\,B\cup C) \ge \mathbb P(A\,|\,B) \end{align} hold?

-
Expanding on @CloseToC's answer, consider the case when $C=B^c$. Then, you are asking whether it is always true that $P(A\mid\Omega) = P(A) \geq P(A\mid B)$ and the answer is clearly No. The conditional probability of $A$ given $B$ can be smaller, larger, or the same as the unconditional probability of $A$. Simple examples of all three cases are easy to find. –  Dilip Sarwate May 19 at 13:32
A related question, where the answer is yes. Does the information content always increase? stats.stackexchange.com/questions/92254/… –  Patrick May 20 at 20:47

Assume $P(B)>0$ to avoid division by zero in the argument below.

Because $B \subseteq B\cup C$, we see that $$P(A\cap B)\leq P(A\cap (B\cup C))\,.$$ On the other hand, $$P(A\cap B)=P(B)P(A\,|\,B)$$ $$P(A\cap (B\cup C))=P(B\cup C)P(A\,|\,(B\cup C))\,,$$ so that $$P(B)P(A\,|\,B)\leq P(B\cup C)P(A\,|\,(B\cup C))$$ or equivalently $$P(A\,|\,B)\leq\frac{P(B\cup C)}{P(B)}P(A\,|\,(B\cup C))\,.$$ In general, the fraction on the right hand side can be any number which is at least one, which shows that everything is possible in the general case, as others have remarked. Only in case $P(B\cup C)=P(B)$ do we get that $P(A\,|\,B)\leq P(A\,|\,(B\cup C))\,.$

Edit: Do see @whuber's reminder below regarding the fact that $P(B)>0$ is a real restriction.

-
In your final conclusion please do not forget you had to assume $\Pr(B)\ne 0$. That assumption (which rules out many practical and interesting cases) was essential; the conclusion is false otherwise. –  whuber May 20 at 20:49
Thanks, I added an edit to emphasize this important fact. –  binkyhorse May 20 at 21:03

It doesn't. Let $X$ be the outcome of a die roll.

$P(X=2|X \text{ is even})=\frac{1}{3}$

$P(X=2|X \text{ is even or odd})=\frac{1}{6}$

However, this isn't a case of gaining more information. I'm fairly certain $H(X|B)\leq H(X|B \cup C)$ where $H$ denotes Shannon information (a measure of information/uncertainty).

-

The answer to your question in the title is yes. More information can cause the probability to lessen.

The statement in the post is not valid, but is satisfieable.

In general; $$\mathbb P(A\,|\,B\cap C) > \mathbb P(A\,|\,B) \iff P(A\cap B\,|\,C)>P(A\cap B)\\ \mathbb P(A\,|\,B\cap C) = \mathbb P(A\,|\,B) \iff P(A\cap B\,|\,C)=P(A\cap B)\\ \mathbb P(A\,|\,B\cap C) < \mathbb P(A\,|\,B) \iff P(A\cap B\,|\,C)<P(A\cap B)$$

For a concrete example consider a deck of card with all heart suits values less than 10 removed. If B is "a card is drawn" and A is "An Ace is drawn" how will the probability of A behaive if you are given the information that a heart is drawn? Or if you get the information that a heart is not drawn?

-

Let me recursively use Bayes theorem for this:

\begin{align}P(A|B \cup C)&=\frac{P(B\cup C|A)P(A)}{P(B\cup C)}\\ &=\frac{P(A|B)P(B)}{P(B\cup C)}+\frac{P(C|A)P(A)}{P(B\cup C)}\end{align}

But we know that $P(B)\leq P(B \cup C)$ and equality only when $C=\emptyset$. So if $C$ is empty or a set with probability measure $0$ then we can always make the statement $P(A|B \cup C)\geq P(A|B)$. Otherwise it is difficult to compare.

-
The basis for the second equality is unclear. I don't believe it is valid because when $A=B=C$ are the entire set it asserts $1=1+1$, which obviously is false. –  whuber May 19 at 15:08
@whuber I believe that the intersection of B and C would have to be empty for the above to hold - otherwise you need to account for the fact that occurrences in the intersection are double counted (which, in the limit of complete sets, leads to your 1=1+1 result which is fine in Boolean algebra but not so hot anywhere else...) –  Floris May 19 at 16:11
@Floris Thanks for the effort, but I'm still baffled. What about when $B$ is empty and $A=C$ is the entire set? Although $B$ and $C$ have empty intersection, the assertion becomes $1=0$. Even the rules of Boolean algebra won't support that! –  whuber May 19 at 16:28
@whuber - you are right, there is definitely a problem with that second step. The second term should be $$\frac{P(C|A)P(A)}{P(B \cup C)}$$ or $$\frac{P(A|C)P(C)}{P(B \cup C)}$$ I think... –  Floris May 19 at 16:43
@Floris Quite possibly. Regardless, it's hard to see how the conclusion holds up. As a (plain vanilla) counterexample, consider a jointly Normal distribution on the $x,y$ plane with unit variances and correlation $1/2$. Letting $A$ be the upper half plane $\{(x,y)\ |\ y\ge 0\}$, $B=\{0,y\}$ (the $y$ axis), and $C=\{1,y\}$ (a vertical line) yields $\Pr(A|B)=1/2$ but $\Pr(A|B\cup C)\lt 1/2$ even though $\Pr(C)=0$. –  whuber May 19 at 19:14