I am trying to understand the derivation of expected loss (equation 2.11 in Elements of Statistical learning) and there is a specific step I do not understand.

We start with

$EPE(f) = E(Y - f(x))^{2}$

and I understand the derivation up to:

$\int_{x} E_{Y \lvert X}(L(x,y)) p(x) dx$

(all omitted steps can be seen here Confused by Derivation of Regression Function)

However, I do not understand how the above is equivalent to:

$E_{X}E_{Y \vert X}L(x,y)$

Why are we multiplying $E_{X}$ by $E_{Y \vert X}$. Where does $E_{X}$ come from? What probability rule/theorem is responsible for this?


1 Answer 1


$E_XE_{Y\mid X}$ is not a multiplication, it's a composition.

If both $X,Y$ are random, then $L(X,Y)$ is a real-valued random variable. By taking the first expectation, you "integrate out" $Y$, which produces a new random variable $$ Z(x) = E_{Y\mid X}\left[L(x,Y)\mid X=x\right]$$ This variable is still random because $X$ is. For each event $X=x$ you have $Z(x) = z$.

As with any random variable, you can take the expectation of $Z$ with respect to $X$: $$ E_X[Z(X)] = E_X\left[E_{Y\mid X}\left[L(X,Y)\right]\mid X\right] $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.