According to the law of total expectation (or iterated expectations), it is true that $E(A|B)=E(E(A|B,C)|B)$.

Why this is not equal to $E(A|B)=E(E(A|C)|B)$?

Why is it important that the first set of information is included in the second one? Can you provide an example to illustrate how important this mistake can be?



Intuitively, $E(A \mid C)$ marginalizes over all information about $A$ that is not contained in by $C$ - including such information that is contained in $B$ but not in $C$. When we take the conditional expectation of $E(A \mid C)$ conditional on $B$, we don't get this "information given only by $B$" back.

To get a simple counterexample, let us take this idea into the extreme and consider a case where $B$ contains all information about $A$ (i.e., tells us the value of $A$) while $C$ contains no information about $A$.

A counterexample

Counterexample: let $A=B$ (with probability 1) and $A$ and $C$ be independent.

Then, we have \begin{equation} E(A\mid B) = E(B\mid B)= B = A \end{equation} However, applying the independence of $A$ and $C$, of the second equation is \begin{equation} E(E(A \mid C) \mid B) = E(E(A) \mid B), \end{equation} and since $E(A)$ is a constant, \begin{equation} =E(A). \end{equation} In general $A$ is not equal to $E(A)$ and thus this shows the second equation in the question is not valid.

Connection of the counterexample to the intuition

Indeed, the results of the counterexample agree with the initial intuition: $B$ carries full information about $A$, thus $E(A\mid B) = A$. Meanwhile, $C$ carries no information about $A$ and thus all this information is lost when taking the expectation conditional on (only) $C$. Therefore, $E(E(A\mid C) \mid B)$ is just $E(A)$, the "prior" expectation of $A$ without conditioning on any extra information.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.