Consider this simple boolean relationship between the binary variables A, B and C:
I.e. $A$ is 1 if either of $B$ or $C$ are 1, otherwise $A$ is always 0.
We also have these extra conditions:
where $PMI(B;C)$ denotes the pointwise mutual information between the "1" outcomes of $B$ and $C$.
Now, unless I'm mistaken (I have limited training in statistics), it follows that:
That is, the amount of pointwise mutual information between the "1" outcomes of $A$ and $B$ is the same as the amount of pointwise mutual info between the "1" outcomes of $A$ and $C$, and is equal to the information-content of the "1" outcome of $A$, i.e. $-log(P(A))$.
But how is it possible for both $B$ and $C$ to both (individually) "completely overlap" with $A$ in terms of their pointwise mutual information, and yet for $B$ and $C$ to not "overlap" with one another at all? When I imagine the meaning of $A=B∥C$ in terms of the pointwise mutual information, this is what seems to make sense:
That is, all of $A$'s pointwise information is "contained within" both $B$ and $C$, and thus $B$ and $C$ must "overlap" via $A$'s pointwise information.
But that obviously doesn't make sense because we've explicitely specified that $B$ and $C$ are independent: $PMI(B;C)=0$ (and this is not made impossible by the relationship $A=B∥C$).
I previously thought that we could say that the "1" outcome of $A$ "contains" some bits (using base 2), and it can "share" those bits with outcomes of other variables (measured via pointwise mutual information), but this example seems to suggest that this simple understanding is wrong/incomplete.
So my question is: Where is the flaw in my understanding of information which causes my intuition to differ from what's obviously true in this simple example?