# Expectation of a function of two variables, not with respect to the joint distribution

I am confused by the following expectation which appears within equation (1) of the following paper http://statweb.stanford.edu/~jhf/ftp/trebst.pdf:

$E_{X}[E_{Y}[L(Y,F(X))]|X]$

I am confused by the inner expectation, which is a expectation of $L(Y,F(X))$ with respect to $Y$. The result would also be $E_{X}[⋅|X]$ which doesn't make sense to me, since it would be the expectation with respect to the distribution of $X$, while conditioning on $X$.

• It is important to note that you have not accurately transcribed the expression in equation 1! You have included an extra closing brace that isn't there. There are problems with mis-matching parentheses throughout that paper. See, for instance, the second equation in section 2--which (by my quick count) is the sixth time some explicit version of this expression has appeared. By comparing the multiple attempts to write down this joint expectation you should be able to figure out what was intended and how to balance the parentheses in any particular instance. – whuber Jan 7 '14 at 16:34
• I agree that there is an extra closing parentheses in the expression in the second equation in section 2, but I don't see an extra closing brace in the expression that I entered. I did however use square brackets for the inner expectation, while the paper uses parentheses. Although the expression in section 2 has an extra parentheses, conditioning the function $L(Y,F(X))|X$ before taking the expectation with respect to Y makes a lot of sense. Despite the typo, is this what the author means? This would imply: $E_{X}[E_{Y}[L(Y,F(X))|X]]$ in place of the expression I transcribed. – tmakino Jan 7 '14 at 17:01
• That's how I read it. The author's meaning is clear because at the very outset he defines this as a bivariate expectation. The version you are looking at is merely one way of expanding it out to emphasize that analyzing the loss can be reduced to analyzing the expected loss (over $Y$, conditional on $X$), as a function of $X$. – whuber Jan 7 '14 at 17:05

Although notation for these matters is not fully standard, the expression

$$E_{Y}\Big[L(Y,F(X))|X\Big]$$ I believe can only be meaningfully translated as (omitting the supports)

$$E_{Y}\Big[L(Y,F(X))|X\Big] = \int f_{Y|X}(y\mid x)\cdot L(y,F(x))dy$$

Then

$$E_X\left(E_{Y}\Big[L(Y,F(X))|X\Big]\right) = \int f_X(x)\left[\int f_{Y|X}(y\mid x)\cdot L(y,F(x))dy\right]dx$$

$$=\int\int f_X(x)f_{Y|X}(y\mid x)\cdot L(y,F(x))dydx$$

$$=\int\int f_{XY}(x,y)\cdot L(y,F(x))dydx$$

$$= E\Big[L(Y,F(X))\Big]$$

I also think Whuber's comments are to the point: the more complicated expression of "one-at-a-time" expected values provides a perhaps more illuminating way to think about this simpler-looking but joint expected value.