Derive P(C | A+B) from Cox's two rules

Question

I am working my way (self-study) through E.T. Jaynes' book Probability Theory - The Logic of Science

Original Problem

Exercise 2.1 says: "Is it possible to find a general formula for $p(C|A+B)$ analogous to [the formula $p(A+B|C)=p(A|C)+p(B|C)-p(AB|C)$] from the product and sum rules. If so, derive it; if not, explain why this cannot be done."

Givens

The rules I have to work with are:

$p(AB | C) = p(A|C)p(B|AC) = p(B|C)p(A|BC)$ and $p(A|B)+p(\bar{A}|B)=1$

Where we can also use logical identities to manipulate propositions. For example: $A+B=\overline{\bar{A}\bar{B}}$

Assumption of Solvability

I believe it must be possible because he does not introduce any other rules later and having a simple logical combination of propositions that was not easily expressible would defeat Jaynes' central thesis. However, I've been unable to derive the rule.

My Attempt

To keep myself from getting confused due to using the same variable names as the givens, I am solving the problem as:

Derive a formula for $p(X|Y+Z)$

Introducing a tautology for conditioning

My best attempt at solving it so far has been to introduce a proposition $W$ which is always true. Thus I can rewrite $Y+Z$ as $(Y+Z)W$ (since truth is the multiplicative identity).

Then, I can write:

$p(X|Y+Z)=p(X|(Y+Z)W)$

So, rewriting one of the givens as Bayes' rule: $p(A|BC)=\frac{p(B|AC)p(A|C)}{p(B|C)}$, I can write:

$p(X|(Y+Z)W)=\frac{p(Y+Z|XW)p(X|W)}{p(Y+Z|W)}=\frac{p(Y+Z|X)p(X|W)}{p(Y+Z|W)}$

Why this doesn't work

The term $p(Y+Z|X)$ is easy to deal with. (Its expansion is referred to in the problem definition.)

However, I don't know what to do with $p(X|W)$ and $p(Y+Z|W)$. There is no logical transformation I can apply to get rid of the $W$, nor can I think of any way of applying the given rules to get there.

Other places I've looked

I've done a Google search, which turned up this forum page. But the author does the same thing I tried without seeing the difficulty I have with the resulting conditioning on the introduced tautology.

I also searched stats.stackexchange.com for "Jaynes" and also for "Exercise 2.1" without finding any useful results.

Answer 1

I am not sure what Jaynes considers to be analogous to $P(A\cup B \mid C) = P(A \mid C) + P(B \mid C) - P(AB \mid C)$ but students have cheerfully used one or more of the following on homework and exams: $$ \begin{align*} P(A\mid B \cup C) &= P(A \mid B) + P(C)\\ P(A\mid B \cup C) &= P(A \mid B) + P(C) - P(AC)\\ P(A\mid B \cup C) &= P(A \mid B) + P(A \mid C),\\ P(A\mid B \cup C) &= P(A \mid B) + P(A \mid C) - P(A \mid BC),\\ P(A\mid B \cup C) &= P(AB \mid B\cup C) + P(AC \mid B \cup C) - P(ABC \mid B\cup C). \end{align*} $$ Do you think any of these are correct?

Note: Changing my (now-deleted) comment into an addendum to my answer, the rules permit the following manipulations: $P(AB \mid C) = P(A\mid C)P(B \mid AC); P(A \mid C) = 1 - P(A^c \mid C).$ The first introduces conditioning on a subset of $C$ but does not eliminate conditioning on $C$. The second also does not eliminate conditioning on $C$. So any manipulations of $P(A\mid B \cup C)$ will always include terms of the form $P(X\mid B \cup C)$, and $P(A\mid B \cup C)$ cannot be expressed in terms of $P(A \mid B)$, $P(A \mid C)$, $P(A \mid BC)$, etc. without including probabilities conditioned on $B \cup C$ also.

Answer 2

For problems like this one, it is sometimes helpful to think less about the formulas and instead draw a picture (in this case, a Venn diagram).

Now stare at the picture and try to visualize what $P(C | A \cup B)$ represents. If you can pick it out of the picture, then you will see that there are several valid ways to write it (two ways jump to my mind off the bat). If you're still stuck, try going back to the usual proof of the ordinary general addition rule for hints.

Remember: a conditional probability concentrates all of its probability mass on the conditioning event (in this case, $A \cup B$). The idea is to focus on the locations where $C$ intersects that event.

By the way, the R code for the figure is

library(venneuler)
vd <- venneuler(c(A=0.2, B=0.2, C=0.2, "A&B"=0.04, "A&C"=0.04, "B&C"=0.04 ,"A&B&C"=0.008))
plot(vd)

Answer 3

You can't get rid of the tautology. I think you are supposed to just add the tautology and apply the product rule and then the sum rule and you get:

$$p(C|(A+B)W) = p(CA|W)+p(CB|W)-p(AB|W) / p(A|W)+p(B|W)-p(AB|W)$$

where all the probabilities are expressed as posteriors to the tautology. I think this is the most similar equivalent to the sum rule that you can get for this problem, so that would be the solution.

Note that if you add the condition $p(AB|W)=0$ (i.e. $A$ and $B$ are mutually exclusive) you get the same expression that you have to prove in the problem 2.2, that would indicate this solution is most probably correct (by Bayesian induction ;).

Answer 4

Bayes Theorem gives $$ p(C\mid A+B) = \frac{p(A+B\mid C)\;p(C)}{p(A+B)} \, . $$ Now, using the conditional and unconditional sum rules, we have $$ p(C\mid A+B) = \frac{p(A\mid C)+p(B\mid C)-p(AB\mid C)}{p(A)+p(B)-p(AB)}\;p(C) \, . $$ Of course, the question is whether or not this formula would be "analogous enough" for Jaynes.