# Law of total expectation and conditioning without including all relevant information

According to the law of total expectation (or iterated expectations), it is true that $E(A|B)=E(E(A|B,C)|B)$.

Why this is not equal to $E(A|B)=E(E(A|C)|B)$?

Why is it important that the first set of information is included in the second one? Can you provide an example to illustrate how important this mistake can be?

## Intuition

Intuitively, $E(A \mid C)$ marginalizes over all information about $A$ that is not contained in by $C$ - including such information that is contained in $B$ but not in $C$. When we take the conditional expectation of $E(A \mid C)$ conditional on $B$, we don't get this "information given only by $B$" back.

To get a simple counterexample, let us take this idea into the extreme and consider a case where $B$ contains all information about $A$ (i.e., tells us the value of $A$) while $C$ contains no information about $A$.

## A counterexample

Counterexample: let $A=B$ (with probability 1) and $A$ and $C$ be independent.

Then, we have \begin{equation} E(A\mid B) = E(B\mid B)= B = A \end{equation} However, applying the independence of $A$ and $C$, of the second equation is \begin{equation} E(E(A \mid C) \mid B) = E(E(A) \mid B), \end{equation} and since $E(A)$ is a constant, \begin{equation} =E(A). \end{equation} In general $A$ is not equal to $E(A)$ and thus this shows the second equation in the question is not valid.

## Connection of the counterexample to the intuition

Indeed, the results of the counterexample agree with the initial intuition: $B$ carries full information about $A$, thus $E(A\mid B) = A$. Meanwhile, $C$ carries no information about $A$ and thus all this information is lost when taking the expectation conditional on (only) $C$. Therefore, $E(E(A\mid C) \mid B)$ is just $E(A)$, the "prior" expectation of $A$ without conditioning on any extra information.