Minimizing the expectation of the loss function Let $X$ be a random variable with density $f_X(x)$. I want to find such $\theta$ that would minimize the expectation of the loss function $\mathbb{E}L(x,\theta)$ where $L(x,\theta) = |x - \theta|$ is a L1 loss function. 
We can rewrite it as $\mathbb{E}L(x,\theta) = \int_{\mathbb{R}} |x - \theta|f_X(x)$. Then find a derivative:$$\frac{d}{d\theta}\int_{\mathbb{R}} |x - \theta|f_X(x) dx = \int_{\mathbb{R}} -\text{sign}(x-\theta)f_X(x)dx$$
But I don't understand what does it mean, how can I find $\theta$ from this expression?
 A: As in many optimization problems, you can proceed by finding $\theta$ such that the derivative is equal to zero. On a side note, what you have is actually a subderivative, since the absolute value is nondifferentiable at $0$.
Set the (sub)derivative to zero:
$$\int_{-\infty}^\infty -\operatorname{sgn}(x-\theta) f_X(x) dx = 0$$
Multiply both sides by $-1$, then split the domain of integration into two halves:
$$\int_{-\infty}^\theta \operatorname{sgn}(x-\theta) f_X(x) dx
+ \int_\theta^\infty \operatorname{sgn}(x-\theta) f_X(x) dx
= 0$$
$\operatorname{sgn}(x-\theta)$ is equal to $-1$ in the first integral (where $x < \theta$) and $+1$ in the second integral (where $x > \theta$):
$$-\int_{-\infty}^\theta f_X(x) dx
+ \int_\theta^\infty f_X(x) dx
= 0$$
Therefore:
$$\int_{-\infty}^\theta f_X(x) dx
= \int_\theta^\infty f_X(x) dx$$
This says that the probability mass to left of $\theta$ is equal to the probability mass to the right of $\theta$. Therefore, $\theta$ is the median of the distribution $f_X$.
A: Another resolution that avoids taking the derivative of the absolute value is to notice that
\begin{align*}\int_{-\infty}^{+\infty} |x-\theta| f_X(x)\,\text{d} x&=\int_{-\infty}^{\theta} (\theta-x)f_X(x)\,\text{d} x+\int^{+\infty}_{\theta} (x-\theta)f_X(x)\,\text{d} x\\&=\int_{-\infty}^{\theta} F_X(x)\,\text{d} x+\int^{+\infty}_{\theta} (1-F_X(x))\,\text{d} x\end{align*}
where $F_X$ is the cdf and then differentiate
$$F_X(\theta)-(1-F_X(\theta))=0$$
leading to$$F_X(\theta)=1/2$$
