Derivation of the conditional median for linear regression in “The elements of statistical learning ” My question is about "The elements of statistical learning" book. I would like to know how to prove that the use of the $L_1$ loss $$L_1: E\bigg[|Y-f(X)|\bigg]$$ leads to have conditional median $\hat{f}(x)=median(Y|X=x)$ as solution to the $EPF(f)$ criterion minimisation in eq(2.11): $$EPF(f)= E_X\bigg[E_{Y|X}\big[(Y-f(X))^2|X\big]\bigg]$$
 A: First, I think you misspelled something in the question. In your case it should be
$$
EPE(f)=\mathbb{E}(\vert Y-f(X)\vert).
$$
What you want to show is that
$$
\text{argmin}_{f \text{ measurable}}EPE(f)=\left(X\mapsto\text{median}(Y\vert X)\right)
$$
This is in fact equivalent to showing that the median is the best constant approximation in the $L^1$-norm, i.e. that
$$
\text{argmin}_{c}\mathbb{E}(\vert X-c\vert) = c^*
$$
where
$$
c^*=\inf\{t:F_X(t)\geq 0.5\}
$$
is the median of $X$ defined via the generalized inverse of the cdf $F_X(\cdot)$ of $X$.This can be easily shown as follows: First assume that $c>c^*$, then
\begin{align}
\mathbb{E}(\vert X-c\vert)&=\mathbb{E}((X-c)\chi_{\{X>c\}})-\mathbb{E}((X-c)\chi_{\{X\in(c^*,c]\}})-\mathbb{E}((X-c)\chi_{\{X\leq c^*\}})\\
&=\mathbb{E}((X-c^*)\chi_{\{X>c\}})-(c-c^*)\mathbb{P}(X>c)\\
&\quad\quad + \mathbb{E}((X-c^*)\chi_{\{X\in(c^*,c]\}})-2\mathbb{E}(X\chi_{\{X\in(c^*,c]\}})+(c+c^*)\mathbb{P}(X\in (c^*,c])\\
&\quad\quad-\mathbb{E}((X-c^*)\chi_{\{X\leq c^*\}})+(c-c^*)\mathbb{P}(X\leq c^*)
\end{align}
Now we bound
$$
-2\mathbb{E}(X\chi_{\{X\in (c^*,c]\}})\geq -2c\mathbb{P}(X\in (c^*,c]).
$$
Hence, we get
\begin{align}
\mathbb{E}(\vert X-c\vert)&\geq \mathbb{E}(\vert X-c^*\vert)+(c-c^*)\left(\mathbb{P}(X\leq c^*)-P(X>c)-\mathbb{P}(X\in (c^*,c])\right)\\
&=\mathbb{E}(\vert X-c^*\vert)+(c-c^*)(2\mathbb{P}(X\leq c^*)-1)\\
&\geq \mathbb{E}(\vert X-c^*\vert),
\end{align}
where we used that $c>c^*$ and $2\mathbb{P}(x\leq c^*)\geq 1$ by the definition of $c^*$. Analogously it can be shown that the same thing holds for $c<c^*$. Hence, we can conclude that the median is in fact the constant RV that approximates $X$ the best in $L^1$.
Finally this can be used to show the final result:
\begin{align}
EPE(f)&=\mathbb{E}(\vert Y-f(X)\vert)\\
&=\mathbb{E}(\mathbb{E}(\vert Y-f(X)\vert\vert X))\\
&\geq \mathbb{E}(\vert Y-\text{median}(Y\vert X)\vert\vert X)\\
&=EPE(\text{median}(Y\vert X))
\end{align}
A: Let's call $(Y - f(X))^2 = g(Y)$. Then, we know that, for continuous cases (for example) 
$$ E[g(Y)] =  \int g(y) f_Y(y) dy $$
And we also know that $$ P(A, B) = P(A|B) P(B)$$  or, 
$$ f_{y, x}(y, x) = f_{y | x}(y | x)  f_{x}(x) $$ 
Then, to derive $E_X \Big [ E_{Y|X} [g(Y) | X ] \Big ]$, we can do:
$$E_x \Big [ E_{Y|X} [g(Y) | X] \Big ] = 
E_x \Big [ \int_{Y} g(y) f_{y | x}(y | x) dy  \Big] \\
\int_{X} \Big[ \int_{Y} g(y) f_{y | x}(y | x) dy  \Big] f_{x}(x) dx
 $$
Which is: 
$$ \int_{X}  \int_{Y} g(y) f_{y | x}(y | x)  f_{x}(x) dy dx $$
$$ \int_{X}  \int_{Y} g(y) f_{y, x}(y, x) dy dx $$
$$ \int_{Y}  g(y) \int_{X}  f_{y, x}(y, x) dy dx $$
$$ \int_{Y}  g(y) f_{y}(y) dy $$
that is the expectation of our $g(Y)$. 
